In the autumn of 2019 I carried out a voluntary project for the Greater London Authority called “Get Straight to the Answer”. The final report is available here:

In September and October 2019 over a 6 week period on a voluntary basis I created a prototype chatbot system for the Greater London Authority.

One of my objectives was to demonstrate my understanding of machine learning, Python programming and Pandas gained from the two Coursera course that I completed in the summer of 2019. The prototype also uses the open source Rasa chatbot framework and runs a Flask web server.

The  prototype implements a chatbot front-end linked to one or more government open data sources at the back-end. The chatbot gathers information from the user and then constructs a Pandas query to retrieve the required answer from the open data source. The business objective of the system is to enable users who are not data specialists to “get straight to the answer” of their question without needing to download, open and investigate a csv or Excel file.

The prototype will be used by the Greater London Authority to evaluate the chatbot approach against other solutions to the same business problem. A reference from the Senior Manager, City Data at the Greater London Authority is available on request. 

The project also provided me with an opportunity to create from scratch a chatbot using Python and Rasa, having already created a chatbot using JavaScript and the Microsoft LUIS natural language processing service for my Master’s Degree dissertation.The prototype is built with these technologies:

Rasa	Chatbot framework	1.2.8
Python	with venv virtual environment	3.6.8
Python requests	HTTP requests library	2.22.0
Pandas	Data analysis library	0.25.1
Flask	Web server	1.1.1
jinja2	Web template engine	2.10.1

The overall system is made up of 4 servers. I commissioned all of these servers in a Windows 10 development environment and also on my own AWS Ubuntu 18 server.

1. Web Server: Flask and jinja2 on port 8080. Based on Rasa webchat FAQ project available under GNU licence (github.com/yogeshhk/GSTFAQChatbot)

To start the server in a Windows 10 venv development environment:

$env:FLASK_ENV = “development”

python -m flask run –host=0.0.0.0 –port=8080 –debugger

2. Rasa NLU and Rasa Core: standard installation with restaurant booking example template running on port 5005

To train and start the Rasa servers in a venv development environment:

rasa train

rasa run -m models –enable-api –cors ‘*’ –debug

3. Rasa Action Server for the custom actions (creating the Pandas query, extracting the response, and returning this to the user) running on port 5055

To start the server in a venv development environment:

rasa run actions –debug

4. Facebook Duckling server. The Duckling server is only used by the Rasa NLU server. This is run as a Docker Container (since the server is written in Haskell) running on port 8000