Add real world data to Linux trace without LTTng hacking

In my 10+ years of embedded Linux work I’ve worked with a lot of Linux trace technologies including the Linux Trace Toolkit (LTTng), System Tap, and even the not so well known and the unlikely to be mainline dtrace for Linux port. These are all very powerful tools that can expose many of the internal details of a Linux system and, depending how you use each tool, create huge volumes of data along the way. I’ve also worked with, so far, three different analysis tools that can take this data and create visualizations of the results.

I’m often asked how to integrate external data into these analysis tools. In this blog post I’ll show you how we do that with our own Sourcery Analyzer. One advantage of this technique is that data can be shown time synchronized with LTTng data without having to become an LTTng expert.

Embedded software developers often need to see how the external world impacts the operation of their software. The best way to see the impact of external events is to show them side-by-side with software execution. Sourcery Analyzer is very flexible in the types of data it can work with. All that is required is an event name, a timestamp, a source, and optionally one or more event payload entries that represent the data. For this example I’m working with data from the OBDII diagnostic port on my Honda Accord. Using an OBDII to USB interface I’ve captured diagnostic data from my car and I’ll show you how to import it to Sourcery Analyzer for analysis.

Preparing the data

To begin lets look at the data. I’m capturing the engine performance information using Movi which outputs performance data as a set of individual files:

Engine RPM.txt
Throttle position.txt
Vehicle speed.txt

Each file has a brief header and then measurements are shown, one per line:

Movi Live Data
2013-01-23  21:02:02

Time(seconds)	rpm
21.405261	825
22.955986	829
24.486217	836
26.151863	837

In addion to LTTng data Sourcery Analyzer knows how to import  a simple text data (EFF, the Event File Format) in this form:

timestamp;event type identifier;event source identifier{;field}{;}
  • timestamp: a decimal integer representing nanoseconds
  • event type identifier: a string which gives the event a name
  • event source identifier: a string which can be useful when multiple event sources are combined in one file
  • field: zero or more fields which hold the actual data. The field format is simple key=value syntax

The file can also contain event declarations specifying event types and data types but for this simple example Sourcery Analyzer can automatically deduce event declarations. All that was required was a simple script to combine the multiple files into one file in the proper format:


Importing to Sourcery Analyzer

Adding the data to Sourcery Analyzer is easy. This 1 minute video will show you how:

Here’s the resulting graph showing RPM over time:

Graph of imported data showing RPM

In the next post I’ll introduce the waveform measurement tool which can give us some useful insight into this graph. If you’d like to try this out on your own then you’ll need:

If you import something interesting to Sourcery Analyzer please come back and leave a comment!

Post Author

Posted April 18th, 2013, by

Post Tags

, ,

Post Comments


About Embedded Blog

Open source experts discuss challenges and community happenings for embedded Linux, automotive, IVI, HMI and technology integration. Embedded Blog


2 comments on this post | ↓ Add Your Own

[…] Add real world data to Linux trace without LTTng hacking […]

Commented on July 14, 2013 at 5:30 am
By mikiliniz wapo

thanks for this

Add Your Comment


October 2015
  • Standardizing across the enterprise
  • September 2015
  • Before Wearables
  • April 2015
  • Why Security in Firmware isn’t Enough
  • Connected meters improve water use, but do they provide owners with false sense of security?
  • Active noise control in next-gen automobiles
  • March 2015
  • Interoperability is Key to AUTOSAR Success
  • February 2015
  • Announced Today: Broadest Embedded Systems Solution for Industrial Automation
  • October 2014
  • Why the FDA Might Claim your Medical Device as DOA
  • September 2014
  • The only component that isn’t decreasing in price
  • Where’s My Data? Internet File System for the Nucleus RTOS
  • August 2014
  • Nucleus Innovate Customer Highlight: Agatsa
  • Debugging Multicore, Vehicle Network Timing & Over-The-Air Software Updates: Some great webinars now available on-demand
  • Heterogeneous Asymmetric Multiprocessing Patient Monitoring Demo Overview
  • July 2014
  • Supporting UIs on Resource Constrained Systems — Guest Post by Phil Brumby
  • “9-1-1: How can I help you…?”
  • June 2014
  • Cybersecurity for Connected Medical Devices and the Internet of Things
  • “I want my IoT”
  • Nucleus Innovate Program Customer Highlight: ZoVolt Ltd.
  • Internet of Things Security: Answering Audience Questions, by Felix Baum
  • Nucleus Innovate Program Partner Highlight:
  • Updating automotive software over the air in the field
  • May 2014
  • New Infotainment Integrations at GENIVI Showcase
  • Guest Blogger Series: Minimizing the Memory Footprint for Internet of Things Devices, by Andrew Caples
  • April 2014
  • Atmel Corporation Joins the Nucleus Innovate Program
  • Addressing the Heartbleed bug and vulnerability in OpenSSL
  • Addressing challenges facing the Internet of Things include power consumption, reliable execution and security
  • What does security for the Internt of Things really mean?
  • New Layerscape architecture based QorIQ multicore processors from Freescale should enable unique home network IoT products
  • Nucleus Innovate Program: When Power Consumption Counts in IoT
  • Guest Blogger: Phil Brumby Q&A on Creating Compelling UIs for Embedded with Qt
  • Live from EELive! Pictures From the First Two Days
  • Open-Source Linux Development for AMD Embedded Processors
  • March 2014
  • Next Week at Freescale Technology Forum: Internet Of Things Connectivity
  • Making use of Bluetooth Low Energy (BLE) profiles for Medical devices
  • Build customized embedded Linux solutions – learn from the experts
  • EE Live In San Jose April 1-3: A Sneak Peek At The Mentor Embedded Booth
  • A methodology for development of high performance embedded software applications
  • Nucleus Innovate Program Customer Highlight: SEVENEY
  • Internet of Things Creates New IP Requirements — Where Does Low Power Fit?
  • This Friday: Automotive Market Disruptions Enabling Innovation – Focus On Open Source
  • Consolidating Automotive Things at Embedded World
  • Video: Internet of Things Connectivity for Embedded Systems at Embedded World
  • Please Join Us March 11th: Security and Performance Benefits of Automotive Virtualization
  • Applying rugged development for internet of things (IoT) cloud based services
  • February 2014
  • Live From Embedded World 2014: Pictures From The First Three Days
  • Debugging performance issues in a Qt application – Part 2
  • Embedded World in Nuremberg: Demos Available at the Mentor Embedded Booth
  • Debugging performance issues in a Qt application – Part 1
  • Thursday 2/27 at Embedded World in Nuremberg: MCAPI, Android and Infotainment, and Automotive Connectivity
  • Embedded World on Wednesday 2/26: AUTOSAR, Automotive Linux & UI, Pre-Silicon and HMIs
  • Power Management & Smart Energy Profile 2.0 Focus on Tuesday Feb 25 at Embedded World in Nuremberg
  • January 2014
  • Finding and Fixing Performance Problems in a Distributed Application with Sourcery Analyzer
  • Putting the smarts in to the smart grid
  • Nucleus Innovate Customer Highlight: Intelligent Power
  • Implementing inter-processor communication on emerging heterogenous multicore SoCs
  • Simulate, Build, Debug and Analyze: Embedded Software Development on a Virtual Prototype with GPU
  • The New Enhanced UI, Responsive Design & Better Search
  • December 2013
  • Demonstrating the use of the TI Jacinto 6 heterogenous automotive processor
  • Join us for an Automotive focused CES 2014
  • Nucleus Innovate Customer Highlight: Reliable Scale Corporation
  • Introduction to the Vista Virtual Prototype Simulator
  • November 2013
  • Using embedded virtualization and ARM TrustZone to meet the functional and security needs of medical and industrial systems
  • Enabling Multi-OS Embedded Systems with Virtualization
  • We are bringing OpenACC to the GNU Compiler Suite!
  • Mentor and CSR announce Bluetooth and WiFi partnership
  • October 2013
  • Come and meet the recently announced Mentor Embedded Hypervisor at ARM TechCon!
  • Pre-Silicon Linux Development with Sourcery CodeBench Virtual Edition and Vista Virtual Prototype
  • Linux in IVI & Telematics Focus at Automotive Linux Summit in Edinburgh
  • This week at IESF in Munich: Focus on Embedded Linux GENIVI & Android
  • This Week at User2User in Munich: Focus on Energy Efficiency
  • Hardware/Software Co-Debug with Sourcery CodeBench Virtual Edition and Veloce Emulation
  • September 2013
  • Nucleus Innovate Customer Highlight: HGM Automotive Electronics
  • Making Automotive IVI Software Work
  • Focus on IVI & GENIVI This Week at IESF
  • AUTOSAR & Network Design at IESF This Week
  • Smart Energy Market Trends: Where are we heading?
  • From the Acorn RISC Machine project to the i.MX – what’s next?
  • Sourcery CodeBench to extend support for embedded development on major MIPS CPUs
  • August 2013
  • Application Layer Trace Visualization Tour with Sourcery Analyzer and Stella, our Nucleus powered self-balancing robot
  • Nucleus Innovate Program – Customer Highlight: Cloud Your Car
  • Determining the Wheel Angle Bias of a Self-Balancing Robot
  • C & C++ Development for custom Instruction Set Architectures
  • July 2013
  • Measuring and Visualizing the Interrupt Latency of a Self-Balancing Robot Running an RTOS
  • Medical Device User Interfaces
  • Nucleus Innovate Program – Customer Highlight: BAB Technologie
  • Developing and Tracing Stella, our Nucleus Powered Self-Balancing Robot
  • Building GNU and LLVM toolchains for custom SoCs
  • The challenges in bringing advanced automotive systems to the masses
  • June 2013
  • Stella, the Nucleus powered, self-balancing robot
  • Debugging Embedded Software – the Do’s and Don’ts
  • Understanding animation performance using Sourcery Analyzer
  • How to make use of Android on Linux IVI systems with Linux Containers
  • Reduce your embedded design compile times with Sourcery CodeBench
  • May 2013
  • Advanced tracing support for ARM based chipsets: what this means for your embedded software development
  • New software trace analysis capabilities in Sourcery CodeBench
  • Simple measurements of software trace data using Sourcery Analyzer
  • Open source components for your embedded platform SDK
  • Smart Energy Profile (SEP) 2.0 specification released – What this means to you?
  • When an Innovative Plan Works!
  • Do you want Embedded Android or Linux in your car?
  • New Supported Boards for ARM & Power Architectures
  • IoT in the Hospital: Webinar by Mentor Embedded, Silex America and Freescale
  • Delivering GENIVI IVI Systems
  • April 2013
  • Pre-Silicon Software Development with Sourcery CodeBench Virtual Edition
  • Enabling M2M devices with zero configuration networking
  • Exercising HW designs using actual software pre-silicon on emulators
  • Using native software development pre-silicon with virtual prototypes to debug software and analyze SoC attributes inlcuding latencies and power consumption
  • Add real world data to Linux trace without LTTng hacking
  • Address your design’s deterministic control and real-time requirements with a RTOS
  • GENIVI Infotainment Solutions – 100 progress meetings
  • “Butter-Smooth” Embedded UIs
  • View the discussion on Linux in automotive and vehicle to vehicle communication – and ask your own questions
  • March 2013
  • Android application development within Microsoft Visual Studio
  • Join Alison Chaiken and Ravi Puvvala on a Google Hangout discussing how Embedded Linux can take on the hard problems of Automotive
  • Leveraging a RTOS to enable current generation MCUs
  • Making use of Android in Automotive Infotainment (IVI) Systems
  • Integrating the MontaVista Automotive Technology Platform – A Progress Report
  • Building a custom embedded Linux distribution with the Yocto Project
  • An Autonomous Vehicle and Voice Recognition in 1982
  • Driving you to distraction
  • February 2013
  • Linux, Android, Automotive Infotainment and Smart Energy – A lot to cover on the Final Day of Embedded World
  • Designing Smart Energy Profile (SEP) compliant embedded applications
  • Learn about Virtual Prototyping, Emulation and Software Debug and USB 3.0 at Embedded World on Wednesday Feb 27th
  • Learn how to Measure RTOS Performance and Design for Lower Power Embedded World 2013 on Tuesday Feb 26th
  • Mentor Graphics Acquires Assets from MontaVista, LLC
  • Free Nucleus RTOS for NXP MCU devices!
  • New supported development boards for ARM architectures
  • A discussion on Battery Life Vs. Delay captured on video
  • Custom toolchains for your custom embedded hardware.
  • Latency: A discussion by Industry experts on the HW and SW considerations for embedded devices
  • January 2013
  • Some examples of connected devices for the home at CES 2013
  • Learning about what’s going on with Automotive at CES
  • Linux will not be Adopted in Automotive
  • This car runs on ice cream
  • Think you can’t afford a commercial RTOS – Think Again!
  • December 2012
  • New Board Support for ARM, MIPS, Power, TI, Freescale, Xtensa, and Coldfire Architectures
  • Migrating from Windows CE to Linux or Android?
  • The role of software in reducing power consumption in embedded systems
  • November 2012
  • An example of multi-core debugging with trace
  • Do try and keep up – GENIVI 3.0 Release
  • Making trace data accessible and useful for embedded developers
  • Driving innovation and creating cool devices through crowdfunding and entrepreneurship
  • Renesas DevCon and the Smart Society
  • October 2012
  • Help! There is a Medical UI Emergency!
  • Connected Cars – Dine in or takeway?
  • September 2012
  • $4.99… Seriously? Here comes the ARM Cortex M4F-based TI LaunchPad
  • Learning about all things Yocto and Open Embedded for Linux
  • Automotive Connectivity in China and India with Freescale silicon
  • LTTng 2.0 and making use of trace tools and techniques to develop complex multicore embedded systems
  • August 2012
  • What is Dynamic Voltage and Frequency Scaling and why should you care?
  • What the Sourcers do to help people make good use of OSS tools for embedded systems
  • To FLOSS, or not to FLOSS – that is the question.
  • New Board Support for ARM, MIPS, Power and Super-H Architectures
  • July 2012
  • Automotive HMI and Driver Safety
  • HMI Joy of Use
  • June 2012
  • Automotive Connectivity and Driver Safety
  • Car makers, Linux and embedded software leaders convene in Detroit