Logging
The ability to log and record drone data is a crucial element in a drone’s mission as well as for debugging purposes. There are different methods for data logging, both over telemetry and onboard on an external device, all of which are covered in this section.
Logging on Linux based OS
The approaches discussed in this section are only possible for systems that run a Linux based OS with access to internal storage space on your autopilot, as is the case with most Parrot drones.
Printing to terminal
Perhaps one of the easiest approaches to logging is to make your module print to a terminal.
You can add your own debug messages by including the <stdio.h>
header to get access to the
fprintf
function. Then, at any point in your code, you can use this function to print your own
message.
<stdio.h>
fprintf(stderr, "<Your debug message>");
It can be useful (if you expect to use this method a lot) to create a macro for this command that
automatically includes additional information on where the message originates, like is done in the
orange_avoider
module.
#define PRINT(string,...) fprintf(stderr, "[orange_avoider->%s()] " string,__FUNCTION__ , ##__VA_ARGS__)
This allows you to only have to write PRINT("<your debug message>")
in your code and it will automatically
prepend "[orange_avoider->%s()] "
to your message, which can help debugging if multiple modules
are printing information simultaneously.
When running in simulation fprintf
automatically prints the message in the Paparazzi Center. You can also
read these messages on the real drone using the following steps:
Open a terminal window (Ctrl+Alt+T).
Make sure you are connected to the drone’s WiFi access point, then open a telnet connection using
telnet 192.168.42.1
.Navigate to the Paparazzi folder using
cd data/ftp/internal_000/paparazzi
.You need to restart the autopilot for the messages to show up. Run
killall -9 ap.elf
, then./ap.elf
to restart the autopilot.When you are finished and the drone is landed, kill the autopilot by pressing Ctrl+C twice, then disconnect using Ctrl+D.
Note
The target address of the telnet
command may be different than what is reported here. You can
double check by going into the Makefile of your drone and looking for the HOST
variable. For
example, the Makefile for the Bebop drone can be found in conf/boards/bebop.makefile
Warning
It is very easy to accidentally start multiple autopilot processes when connected to the drone by
WiFi. This can cause flickering in the GCS as it receives duplicate messages, as well as slowing down
the execution time. You can terminate all autopilot processes simultaneously by running killall -9 ap.elf
File Logger
Logging data onboard on a Linux based OS is pretty straightforward. The Bebop drone, for example, is
equipped with onboard storage, which can be used to store the logs. First, it is necessary to add the
logger_file
module in your airframe and specifying the path to the internal storage.
<firmware name="rotorcraft"/>
<module name="logger_file">
<define name="FILE_LOGGER_PATH" value="/data/ftp/internal_000"/>
</module>
</firmware>
Then, in the C file sw/airborne/modules/loggers/file_logger.c
add the header of the module that
contains the variables that you want to log. Within file_logger.c
you can specify which variables
you want to have logged in the same fashion as normal fprintf
as described in the previous sesction.
The variable names and types have to be specified.
The file logger needs to be manually started and stopped by going to the Paparazzi GCS. In the module
settings tab navigate to Settings -> Modules. Here you can start and stop a log. To retrieve the log file
connect to the drone by WiFi and using either a browser or an FTP client (like the Linux standard Files application)
connect to ftp://196.168.42.1
. This will take you to the FTP folder of the Bebop. Browse to /Internal000/
.
Here we can find the .csv files, named 00000.csv
to 0000X.csv
depending on the flights you logged.
Copy the files to your computer for analysis.
Logging on ChibiOS based OS
FlightRecorder
Logging on ChibiOS systems can be achieved using the flight_recorder
module, which depends on a
few other modules that also need to be included in your airframe.
<firmware name="rotorcraft">
<module name="flight_recorder"/>
</firmware>
Data is stored on an SD card using the pprzlog
format, discussed in more detail in the next section.
The logging starts automatically soon after the drone is connected to power. The logger status is reported
on the LOGGER_STATUS
message, which can be viewed using the GCS by opening Tools -> Messages.
The flight_recorder
will record all messages that are specified within a FlightRecorder
process
that should be included in your airframe’s telemetry file.
<telemetry>
<process name="FlightRecorder">
<mode name="default">
<message name="ATTITUDE" period="0.05"/>
<message name="IMU_ACCEL" period="0.02"/>
<message name="IMU_GYRO" period="0.02"/>
<message name="IMU_MAG" period="0.02"/>
<!-- etc. -->
</mode>
</process>
</telemetry>
These telemetry files are located within the conf/telemetry/
folder, and are associated to a specific
airframe in the conf.xml
. For example, conf/userconf/tudelft/conf.xml
specifies for a default Bebop:
<conf>
<aircraft
name="Bebop_default"
ac_id="20"
airframe="airframes/examples/bebop.xml"
radio="radios/dummy.xml"
telemetry="telemetry/default_rotorcraft.xml"
flight_plan="flight_plans/rotorcraft_basic.xml"
settings="settings/rotorcraft_basic.xml settings/control/rotorcraft_speed.xml"
settings_modules="modules/ahrs_float_mlkf.xml modules/air_data.xml modules/bebop_ae_awb.xml modules/bebop_cam.xml modules/geo_mag.xml modules/gps.xml modules/guidance_rotorcraft.xml modules/imu_common.xml modules/ins_extended.xml modules/nav_basic_rotorcraft.xml modules/stabilization_int_quat.xml modules/video_rtp_stream.xml"
gui_color="#ffffbc3bce5b"
/>
</conf>
Logger SD ChibiOS
The flight_recorder
module is essentially a wrapper for another logger module called logger_sd_chibios
,
which handles the log file creation/closing. The flight_recorder
uses these functions to open a log file and
outputs to it a telemetry message (in binary) that by defaults contains all the messages specified in the telemetry XML
as discussed in FlightRecorder. However, if more control over what actually goes into the log file is necessary
or desired, it is possible to directly use logger_sd_chibios
to create custom log functions. First, the module
must be included in your airframe.
<firmware name="rotorcraft">
<module name="logger" type="sd_chibios"/>
</firmware>
Then, in the .c file where you want the data to be logged from, you can create custom log functions that will write to the log either in binary format, or directly in ASCII format. To write directly in ASCII:
#include "modules/loggers/sdlog_chibios.h"
static inline void custom_log_function_ascii(void) {
// Check that log file has been created correctly
if (pprzLogFile != -1) {
// Write whatever you want to this file using sdLogWriteLog()
sdLogWriteLog(pprzLogFile, "<Your log message %f %f>", foo, bar);
}
}
To write in binary instead, you need to create a function that behaves in a similar way as the FlightRecorder, which sends telemetry data directly to the log file instead of over the air.
#include "modules/loggers/sdlog_chibios.h"
#include "modules/loggers/pprzlog_tp.h"
// Any log file could be specified from the airframe
// Set the default to the one created by logger_sd_chibios
#ifndef MY_LOG_FILE
#def MY_LOG_FILE flightrecorder_sdlog
#endif
// Create a function that sends all your desired messaged to the log
static void custom_telemetry_send(struct transport_tx *trans, struct link_device *device) {
// There can be more than one pprz_send function
pprz_send_msg_YOUR_MSG(trans, device, AC_ID, &foo, &bar);
}
static bool log_tagged;
static inline void custom_log_function_binary(void) {
if (MY_LOG_FILE.file != NULL && *(MY_LOG_FILE.file) != -1) {
if (log_tagged == false && GpsFixValid()) {
// Write at least once ALIVE and GPS messages
// to log for correct extraction of binary data
DOWNLINK_SEND_ALIVE(pprzlog_tp, MY_LOG_FILE, 16, MD5SUM);
// Log GPS for time reference
uint8_t foo_u8 = 0;
int16_t foo_16 = 0;
uint16_t foo_u16 = 0;
struct UtmCoor_f utm = *stateGetPositionUtm_f();
int32_t east = utm.east * 100;
int32_t north = utm.north * 100;
DOWNLINK_SEND_GPS(pprzlog_tp, MY_LOG_FILE, &gps.fix,
&east, &north, &foo_16, &gps.hmsl, &foo_u16, &foo_16,
&gps.week, &gps.tow, &utm.zone, &foo_u8);
log_tagged = true;
}
// Send custom telemetry function directly to log
custom_telemetry_send(&pprzlog_tp.trans_tx, &(MY_LOG_FILE).device);
}
}
And finally, include your custom log function in your module periodic function (or whatever other place that should trigger the log writing).
void your_module_periodic(void) {
// If logging in ASCII
custom_log_function_ascii();
// If logging in binary
custom_log_function_binary();
}
Decoding FlightRecorder logs
To download and convert the data, you need to first connect the SD card to your laptop, navigate to the
FLIGHT_RECORDER folder and transfer to your computer the relevant logs, named fr_XXXX.LOG
. To convert
this binary file into Pprzlog format files the program sw/logalizer/sd2log
has to be used.
Make sure the environment variables are set before running Paparazzi executables from the commandline.
They can be set in your Terminal by using
export PAPARAZZI_HOME=~/paparazzi
export PAPARAZZI_SRC=~/paparazzi
Tip
You can add the export
lines above to your .bashrc
file to automatically set the environment variables
every time a new Terminal is opened.
Once the environment variables are set you can run sd2log
in your terminal:
~/paparazzi/sw/logalizer/sd2log fr_XXXX.LOG # Output stored in var/logs
~/paparazzi/sw/logalizer/sd2log fr_XXXX.LOG <output_dir_path> # Output stored in <output_dir_path>
This will produce the Paparazzi .log, .data, and .tlm files that are stored in var/logs
. It creates a timestamp from the .tlm and changes the filename
to the take-off time if a GPS mesage with correct time was available in the file, or the current local PC time if no
GPS was available. The .log file will be recreated either from the current configuration or the MD5-labeled files that
are stored in var/conf
each time you build an aircraft.
Note
For the decoding process to work properly you must run sd2log
from the same folder that contains the
messages.xml
file used during compilation of the aircraft. This is often not an issue, but you may run into
problems if you work with multiple branches within your paparazzi repository.
Pprzlog format
The Pprzlog format creates multiple files that can be used to analyse flight data and replay the flight. A log is split into the following files:
A
.log
file, an XML file, which contains a copy of the whole configuration (airframes, flight plans, …)A
.data
file, an ascii file, which contains the list of the received messages. Each message is time-stamped in seconds since the creation of the file and marked with the ID of the sending aircraftA
.tlm
file, a copy of the original.LOG
file renamed with the same name as the.log
and.data
file to make it easier to associate the original log with the decoded files
Tip
Because sd2log
outputs a copy of the original .LOG
file, the decoding process can be called directly
on the file still on the SD card while it’s mounted on your computer.
The name of the files associated to a specific log is the same, and is generated from the date and time of creation.
The lines of the data
file are formatted according to the message description listed in the conf/messages.xml
file. For example:
30.5941 186 ATTITUDE 0.036228 0.018550 0.021443
contains an ATTITUDE
message received at time 30.5941s, from aircraft 6. According to the ATTITUDE message
description:
<message name="ATTITUDE" id="6">
<field name="phi" type="float" unit="rad" alt_unit="deg"/>
<field name="psi" type="float" unit="rad" alt_unit="deg"/>
<field name="theta" type="float" unit="rad" alt_unit="deg"/>
</message>
In this case, at the time the message was logged, the attitude of the drone corresponded to \({\phi} = 0.036228\),
\({\theta} = 0.018550\), and \({\psi} = 0.021443\) radians. Note that the appropriate messages.xml
description, i.e. the one which has been used while the log was created, is itself stored in the associated .log
file. It may differ from the current one in your conf/
folder.
Note
The .data
files may be huge. They can be efficiently compressed, with the bzip2
compression format seemingly
performing better than others on these files.
Data plotting
There are different methods to visualize and process the data stored in log files.
Log Plotter
If no post-processing of the data is required, log data can be visualized using the Log Plotter, located in
sw/logalizer/logplotter
. It can be launched either from the command line, or through the Paparazzi Center by
navigating to Tools -> Log Plotter. This tool can plot data from the same or different logs in the same window, as
well as offering the option to export the track as a KML file for Google Earth, or to a CSV file for further data
processing.
Log File Player
A flight can be replayed with the Log File Player (sw/logalizer/play
), which can be started either from the
command line, or from the Paparazzi Center by navigating to Tools -> Log File Player, or even from the Session selection
box to start a complete replay session with the GCS, server, and player tool. In this last case, this agent then
acts as a substitute for the Data Link agent and will send onver the bus the messages that had been sent by the aircraft
while the log was recorded.
Note
While replaying a log through the GCS, it is a good idea to disable a new log creation from the Server. This can
be achieved by launching the Server process with the -n
option.
While doing a log replay it can be very valuable to launch a Messages process (Tools -> Messages). This allows for the use of the Tools -> Real Time Plotter and also displays all the data received from the aircraft.
If the Log File Player is launched with the -o
option, the player will send to a serial port all the binary
messages as they had been received through the modem during the flight. Additional options include -s
to set
the baudrate (default 9600), and -d
to set the device (default /dev/ttyUSB0
).
Paparazzi Log Parsing
The Github repository tudelft/paparazzi_log_parsing provides handy Matlab and Python tools to convert the log data into a dictionary-like structure for easy post-processing. The Matlab and Python scripts are very similar and function in almost the same way except for certain language-specific functionality, and contain an example file that illustrates how to use the functionalities provided by the repositories.