Trufi Code Walkthrough Notes - Back End

Mar 5, 2022

Some background information        1

Using the Command Line        1

SSH - what is it?        2

Setting up Everything        2

Inside the Folder        2

Preparing the Terminal        3

Preparation to Execute the builders        3

Executing the builders on the Command Line        4

Trufi Server Overview and Extensions        4

Installing/Uninstalling an extension and running the back end afterwards        5

Some background information

• Back end is 2 parts

• Services needed to run the back end
• Building scripts
• The back end just serves the data and the data needs to be built
• We work with Linux - and everything is optimized for Linux
• Windows is not supported because they are not usually used on servers
• Get used to Linux and get used to the command line! This is because you won’t have a GUI later to work with, so best to get used for this now
• You can set up Ubuntu on a virtual machine (good for beginners)

• The terminal

• Nothing special but not from everyday life (maybe!) :)
• Terminal is not always intuitive, unlike a GUI

• Paths - you will need to pay attention to these, but don’t worry, it will be ok :)

Using the Command Line

• This is really good to get used to
• You can also find the history and the past bash history
• Use the up and down arrows

SSH - what is it?

• You can execute it on Windows or whatever you use (Linux, Mac, Windows)
• If you pass ssh and then your username, so for example yourname@servername
• You can also write your own scripts with SSH to log into a server - it really depends on your workflow and how you like to do things

Setting up Everything

• We are on a virtual machine and we imagine that this is our development server. It might have a GUI, might not
• First, Sören creates a folder and he puts SSH into that server
• SSH is a command like standard to get into servers and administrate them from the command line
• It’s included in Window from 8+
• First we must install the required files

• Sudo - we use this to execute as a super user or administrator (in windows terms)
• It always you to not get certain errors while working in the command line
• First do “Sudo apt install git docker docker.to docker-compose”

• Docker makes it easy to make containers
• For security reasons
• To simplify things

• Next, we grab the resource from Trufi github by cloning it to your computer

• This has everything you need to run our services and servers (INSERT URL TO GITHUB HERE)
• Once you download LS and then CD into the folder
• Hitting tr and then tab make it auto-complete and then you are in!
• It is not recommended to execute this on your production server but to build the environment on your personal computer or development machine (except if you plan to automate the whole thing, but maybe don’t do that first)

Inside the Folder

• We have buildings and each builder has to do something  
• The folders are called name-builder - each folder represents one builder
• You can read the how-tos for each builder as well
• You can also read the README in this folder and it explains the requirements and step by step what to do as well as the concept of the configuration files
• You can use these for multiple cites!
• We need this when we are paid to do this for cities

• We are going to create an environment file for our city
• To do this, you close the file and then rename it (make sure to make all your conventions the same (name of the country, then name of the city)
• Then we can look at both the two files and compare the structure

• There are for and when variables
• Shortcode for the city
• Bounding box (it's a way to specify extraction, also used in OSM)

• You can read info for how to do this in the README tutorial, as well as how to get the bounding box
• You can use a tool to help you build a bounding box

• The bounding box tool
• It runs in your web browser

• You can move around and change orientation on the map

• Once you have the bounding box, you need to name it

• You copy it and then bring it back to your file and remove the other text
• Make sure to remove any white space, or otherwise it will cause errors

• We then need to modify the geofabrik path

• Geofabrik is a german company
• They have lists of regions
• When you click in the region you can click on the city that you want
• Under commonly used formats, choose the osm.pbf file and copy the link
• Then remove the string in your old file and replace it with what you got from Geofabrik

• The back end at Trufi was also created by Samuel, so if Sören does not know the answer to your question, try asking Samuel !
• Once you validate all is well in your file, save it and close it

Preparing the Terminal

• You can do this by copying and pasting
• Take advantage of the auto completion feature
• It executes in the bash

Preparation to Execute the builders

• In the README, you can see a list of the builders that you can execute in order.
• Make sure that you do them in order since each one has dependencies on the one above BUT you don’t need to execute all of them, it depends on which city you are working with.
• Some cities and countries run on formal traffic - bus stops with fixed timetables
• In that case, you don’t need to execute the GTFS builder because you already have data from your government (most likely, anyways)
• If your authority does not provide this, you should execute this builder
• Fixed routes

• Fixed stops
• Always goes in same order
• Shows up at a designated time

• If you don’t need to run them and you run them, you might end up getting some errors. Make sure to evaluate your project and see if it's useful for you before running them
• It also depends on the budget that you have for a server
• The builders all have a hyperlink which will bring you to the wiki of each of them so that you can get more information to see if you need to run them

• These will be based on a template and will talk about the builder and it’s dependencies

Executing the builders on the Command Line

• Cd into the builder folder in the trufi repo that you downloaded to your machine earlier
• Copy and paste and save yourself time :)
• You go into the builder repo, you go into its read file and paste it into the command line
• In the data folder, all the output of the commands will be stored  

• An otp folder will be there - it's an extensions
• You want to copy the contents of this folder and put it into the extensions folder

• This uses the bounding box that we specified earlier as well as the environment file

• Make sure that you follow the “how to use” in the builders, because it tells you everything you need to know about each specific builder, their dependencies, and anything else you need to copy, etc etc etc.
• If you don’t know what or how to execute, you can always ask in the tech-team channel in our Slack
• There might be cases where you have only read access, so you might need to finagle some things
• The way that Docker works is that data contains a bunch of (bounds?) The container is inside of a docker container
• In Linux, things are directly integrated so you might see folders “mounted.”
• There might be data structures that we need to organize on our own

Trufi Server Overview and Extensions

• We have a structure with a bunch of scripts, each is documented
• But we also have folder and the most important folder here is the extensions folder
• Each extension represents a single service (the back end is a bunch of services)

• The routing (the app needs internet connection)
• The map background (the app is not connected to openstreetmap, we run our own (map tiles)
• This is because we then have more independence and can style it as we want
• The trufi servers are connected to each other
• You can see which builders each extension depends on

• We can also copy paste from our Trufi Server folder - copy the config file that we created
• We use various otp versions (it depends on where the app is located)
• We recommend that you use the other option – there is a list of versions by going to the extensions folder and then OTP folder
• You can then choose the one that works best for your country (whether or not you have formal or informal transit) - Sören needs to document this but for now just ask in the tech-team Slack channel
• You can then configure things by running scripts (some are in the folders, or you can run your own, depending on your needs)

Installing/Uninstalling an extension and running the back end afterwards

• To install - its all documented in the extensions folder
• In the development folder
• You install an otp extension and that modifies some Docker files
• Some things are added to the script automatically
• Because it involves Docker, you need to execute
• The containers are then build and the containers are run
• You can also remove extensions with a script