Setting up AirSim with Unreal Engine 5.1.1

Mayan Iyer, Srijan K. Pal and Jiarong Hong

Mechanical Engineering & Saint Anthony Falls Laboratory, University of Minnesota

Windows 10

• Installations -
• Download Epic Games and setup an account (Link)
• Download and install Unreal Engine 5.1.1
• Install Visual Studio 2022. Make sure to select

Desktop development with C++

Windows 10 SDK 10.0.19041

Latest .NET Framework SDK under ‘Individual Components’ (should be there on top)

[These can be downloaded later after installing Visual Studio 2022 using Visual Studio Installer and then going into ‘Modify’]

• Building AirSim -
• The following steps creates the required plugins in Unreal/Plugins

[Colosseum repo (for UE5.1.1) can be used in place of AirSim repo (Colosseum preferred)]

• Start the ‘Developer Command Prompt for VS 2022’ as admin

Change directory to D:

git clone https://github.com/CodexLabsLLC/Colosseum.git (preferred)

(or) git clone https://github.com/Microsoft/AirSim.git;

cd Colosseum; (or) cd AirSim;

build.cmd;

​

Unreal Engine 5.1.1

• Changing the Vehicle Model
• Download DroneModel.fbx (Link)
• Click on ‘Settings’ in the top right of the Content Browser of UE, and select the ‘Show Plugin Content’ option
• In the Content Browser, navigate to Plugins -> Blueprints, create a new folder here and import the DroneModel.fbx file
• Open ‘BP_FLyingPawn’ (blueprint) in ‘Plugins\Blueprints’ by double clicking it.

Go to the ‘Viewport’ tab. In the ‘Components’ window, select the ‘BodyMesh’.

In the ‘Details’ window change the static mesh to the one from the DroneModel.fbx that was imported.

• Adjust the position of Prop0-Prop3 to correspond to the rotors of the quadcopter.

Change material (color if required)

​

• Computer Vision Mode (Optional)
• settings.json can be edited to have SimMode: ComputerVision
• This mode can be used to collect data (images) from the simulation environment. User can freely move the camera around in the environment to capture the images

​

Windows Subsystem for Linux (WSL2)

• Install Windows Subsystem for Linux (WSL 2) with Ubuntu 18.04 (Link)
• ROS Setup:
• Install gcc>=8
• Install ROS melodic (Link)
• Install tf2 and mavros with the command in a WSL 2 terminal -

sudo apt-get install ros-melodic-tf2-sensor-msgs ros-melodic-tf2-geometry-msgs ros-melodic-mavros*

• Install catkin tools with the command -

pip3 install "git+https://github.com/catkin/catkin_tools.git#egg=catkin_tools"

• Build AirSim (Colosseum) again:

git clone https://github.com/CodexLabsLLC/Colosseum;

cd Colosseum;

./setup.sh;

./build.sh;

gc;

source ~/.bashrc;

• And build the ROS package:

cd ros;

catkin build;

AirSim and WSL 2 Integration

• Connecting WSL to Windows over the network

The network access is enabled using this command in WSL.

• Open the .bashrc file -

nano ~/.bashrc;

• Add this line at the end of the .bashrc file–

export WSL_HOST_IP=$(ip route | awk '/^default via/ {print $3}')

• Save the changes and close the .bashrc file. Then source the .bashrc file -

source ~/.bashrc;

​

• Deactivating Windows Firewall for WSL

Windows firewall needs to be deactivated for WSL with this command in PowerShell

(Change – Profile to Domain or Private depending on the type of network) –

Set-NetFirewallProfile -Profile Public -DisabledInterfaceAliases "vEthernet (WSL)"

​

​

​

AirSim and WSL 2 Integration (contd.)

• For every terminal opened, the ROS setup needs to be sourced-

(Enter this command in every terminal opened or add this line to the ~/.bashrc)

source <PATH_TO_COLOSSEUM>/ros/devel/setup.bash;

(Replace <PATH_TO_COLOSSEUM> with the your path to Colosseum directory. The command should be similar to: source ~/Colosseum/ros/devel/setup.bash; )

​

• Ensure that the settings.json is updated in /Documents/AirSim (in Windows Link) to include all the required sensors that support the ROS application
• List of useful settings files (Link). The files in the link are in .txt format. Copy paste the text in settings.json file in the /Documents/AirSim
• For initial testing you can use the - front_stereo_and _center_mono.txt
• For single drone autonomous smoke tracking use - single_drone_smoke_track.txt

​

AirSim and WSL 2 Integration (contd.)

• There are 2 project files which contain UMore Park and Cedar Creek Environment in the (Link).

(for single drone autonomous smoke tracking open the UMore Park)

​

• Launch the Unreal Engine project file and play the simulation
• Launch AirSim node using command (in WSL 2 terminal)

(this node needs to be launched in a terminal every time connecting to UE; use instead of roscore) -

roslaunch airsim_ros_pkgs airsim_node.launch output:=screen host:=$WSL_HOST_IP

​

• Teating: Launch Rviz environment with command - roslaunch airsim_ros_pkgs rviz.launch

​

​

AirSim with PX4 SITL and MAVROS in WSL 2

Steps to integrate PX4 SITL and MAVROS with AirSim

• Clone the PX4-Autopilot git repository -

git clone https://github.com/PX4/PX4-Autopilot;

• Change the version to v1.12.1 –

cd PX4-Autopilot;

git checkout v1.12.1;

• In the Colosseum/AirSim ros folder, install MAVROS by following the ‘build from source’ instructions here. The ~/catkin_ws/src could be your Colosseum(or AirSim)/ros/src. Make sure you git checkout to version 1.12.1 of MAVROS. You can follow the following commands -

cd <PATH_TO_COLOSSEUM>/ros;

catkin init;

wstool init src;

sudo apt-get install python-catkin-tools python-rosinstall-generator -y;

rosinstall_generator --rosdistro melodic mavlink | tee /tmp/mavros.rosinstall;

rosinstall_generator --upstream mavros | tee -a /tmp/mavros.rosinstall;

AirSim with PX4 SITL and MAVROS in WSL 2 (contd.)

(Remaining Commands continued from the last slide -)

wstool merge -t src /tmp/mavros.rosinstall;

wstool update -t src -j4;

rosdep install –from-paths src –ignore-src -y;

sudo ./src/mavros/mavros/scripts/install_geographiclib_datasets.sh;

catkin build;

source devel/setup.bash;

​

• Open cmd in windows and enter ‘ipconfig’.

Get the IPv4 of WSL2. You need to export this IP by adding a line in .bashrc file –

nano ~/.bashrc;

Then add this line at the end (replace <IP> with the IPv4 of WSL 2 you get from above step)

export PX4_SIM_HOST_ADDR=<IP>

Save, exit and source the .bashrc file- source ~/.bashrc;

Swarm Setup

Steps to launch swarm of drones in the Unreal Engine Simulation

• Open incoming TCP port 4560, 4561… n and consecutive incoming UDP port 14540, 14541,... n using your firewall configuration (where n = number of drones on the swarm)
• Ensure that the settings.json is updated in /Documents/AirSim to include the settings as shown in the ‘two_drone_swarm.txt’ file [Location: Research projects\Report2023-_Autonomous Drone for Dynamic Smoke Plume Tracking\Data\Unreal Engine Simulation\AirSim Settings]
• The above settings file is for 2 drones, similarly add as many drones as you want in the settings.
• Launch the Unreal Engine project file and play the simulation.

You should see a message on the top left saying ‘Waiting for TCP connection…’

• In WSL2 new tab, change to PX4-Autopilot directory and run -

cd PX4-Autopilot;

./Tools/sitl_multiple_run.sh n;

(Where n is the number of drones in the swarm)

Example for 2 drones (n=2) run - ./Tools/sitl_multiple_run.sh 2;

Swarm Setup (contd.)

​

• In separate WSL2 tabs, run these Mavros commands separately. Run n commands for each separate drone (running mavros for each drone instance) -

roslaunch mavros px4_swarm.launch ns:=drone1 fcu_url:=udp://:14030@127.0.0.1:14030 tgt_system:=1;

roslaunch mavros px4_swarm.launch ns:=drone2 fcu_url:=udp://:14031@127.0.0.1:14031 tgt_system:=2;

…

roslaunch mavros px4_swarm.launch ns:=drone1 fcu_url:=udp://:1403n-1@127.0.0.1:1403n-1 tgt_system:=n;

(Where n is the number of drones)