Monitoring and collecting syslog messages from Unix Domain Socket

Application log is the traditional way to monitor an application/service. On *nix-based system, Syslog is a common but powerful tool for centrally monitoring applications logs. The primary use of syslog is for system management as capturing log data is critical for sysadmin, devOp team, or system analysts, etc. This log data is helpful in case of investigating/troubleshooting problems and maintaining healthy functioning of systems.

Syslog offers a standard log format and a standard alert system with different severity levels to applications in form of a log API. Log daemons such as rsyslog are versatile and flexible with various configuration options that enable different way to interact with the logs: log to file, log to a remote server via network (TCP, UDP sockets), log to local Unix domain socket. Log clients or log analytic applications can collect log data from the log daemon via these interfaces.

Although it is feasible to directly read log messages from the regular syslog output files, it is more preferable to collect log data from the daemon using the socket interface since socket is more suitable for data streaming. TCP/UDP sockets can be used to access log data from the network (TCP/IP). But if the application runs locally on the same machine as the log daemon, Unix domain socket (UDS) may be the best option.

Unix Domain Socket is an inter-process communication mechanism that allows bidirectional data exchange between processes running on the same machine. Thus, UDSs can avoid some checks and operations (like routing); which makes them faster and lighter than IP sockets.

In this post, we will learn how to collect log data from syslog via UDS in C. We will use rsyslog as log daemon in this post.

A use case will be presented at the end of the post.

13/06/2021 release, AntOS, SDK, documentation

AntOS v1.2.0-beta release

After a long testing period, AntOS v1.2.0-beta is now released!!!

Change logs

  • Improvement GUI API
    • [x] File dialog should remember last opened folder
    • [x] Add dynamic key-value dialog that work on any object
    • [x] Window list panel should show window title in tooltip when mouse hovering on an application icon
    • [x] Allow pinning application to system panel
    • [x] Improvement application list in market place
    • [x] Allow triplet keyboard shortcut in GUI
    • [x] CodePad allows setting shortcut in CommandPalette commands
    • [x] CodePad should have recent menu entry that remember top n file opened
    • [x] Improve File application grid view
    • [x] Label text should be selectable
    • [x] switch window using shortcut (CTRL+ALT+1, CTRL+ALT+2)
    • [x] Loading bar animation on system panel
    • [x] Multiple file upload support
    • [x] Generic key-value dialog
    • [x] Add bootstrap font support for icons
    • [x] Classify applications by categories in start menu
    • [x] Support vertical and horizontal resize window
  • MarketPlace now classifies application by categories
  • CodePad is no longer default system application, it has been moved to MarketPlace
  • More applications added to MarketPlace
  • Antos SDK
    • SDK is no longer included in Antos base release, it can be installed via MarketPlace
    • The SDK now has a generic API that can be used in different development tasks other than AntOS application
    • Heavy SDK tasks are now offloaded to workers
    • Introduce new JSON based syntax for SDK task/target definition
  • From this version, docker image of All-in-one AntOS system is available at: https://hub.docker.com/r/xsangle/antosaio

Demo

A demo of the VDE is available at https://app.iohub.dev/antos/ using username: demo and password: demo.

If one want to run AntOS VDE locally in their system, a docker image is available at:
https://hub.docker.com/r/xsangle/antosaio/

AntOS applications (Available on the MarketPlace)

https://github.com/lxsang/antosdk-apps

Documentation

Data visualization: global view of blog posts relationship

As stated on a post where i talked about using tf-idf to detect similarity between two blog posts, my blog is just a bunch of posts sorted by date, no category, no fancy features like user interest tracking, post ranking, etc. I usually work on many different domains (robotic, IoT, backend, frontend platform design, etc.), so my posts are mixed up between these domains. This may be difficult for readers who want to follow up their interesting topic on my blog.

So what is a good strategy for navigating between posts on a blog ?

sysmond: Simple service for (embedded) Linux system monitoring

Working on my DIY robot software (Jarvis) in headless mode, i came across a situation where i needed to monitor the system resource such as CPU, battery, memory, network and temperature to measure the "greedy" of my robotic application. Furthermore, as the robot was battery powered, battery safety was a real concern, so i needed something to monitor the battery and shutdown the system when the battery was low to protect it from falling bellow the usable voltage range.

So i've searched for an application/service that allows me to:

  • Monitor system memory, CPU, storage usage and temperature
  • Monitor network consumption
  • Monitor the robot battery and power off the system if the battery is low

None of existing applications/services satisfy all of these requirements, especially, the battery monitoring feature. So i've decided to write a small service that i called sysmond.

sysmond is a simple service that monitors and collects system information such as battery, temperature, memory, CPU, and network usage. The service can be used as backend for applications that need to consult system status. Although it is a part of Jarvis ecosystem, sysmond is a generic service and can be easily adapted to other use cases.

https://doc.iohub.dev/jarvis/asset//c_2/s_5/jarvis_monitoring.png

Example of AntOS web application that fetches data from sysmond and visualize it as real-time graphs on my Jarvis robot system. Detail on the use case can be found here

Sysmond monitors resource available on the system via the user space sysfs interface provided by the linux kernel.

JETTY: Jarvis Serial to ROS-2 Transport Layer

My ROS based DIY robot( presented in the previous post) uses the NVIDIA Jetson Nano for high level robotic algorithms with the ROS 2 middle-ware. The Jetson is connected to an Arduino via a serial link for low-level hardware interaction and control.

As the Arduino is used for low-level communication with actuators/sensors. We need a software transport layer on top of the physical serial link (Jetson - Arduino) to stream (sensor) data/command from Arduino to ROS 2 and vice versa. On Dolly (my previous robot version), which used ROS 1, this was handled by Rosserial, a protocol for wrapping standard ROS serialized messages and multiplexing multiple topics and services over a serial link. On ROS 2, however, Rosserial is not available. Other alternative solutions exist but are not mature enough, some implementations require more computational resource which exceeds the capability of the Arduino Mega 2560.

So i decided to implement a dedicated transport layer for Jarvis called JETTY (Jarvis SErial to ROS-2 TransporT LaYer). I do not aim at a generic protocol for ROS to serial communication like ROS serial. Instead, the implementation of the transport layer should be specific only to the robot. However, the protocol must be easy to extend to adapt to any future upgrade of the robot such as adding more sensor/actuators.

Requirements on the transport layer:

  1. The transport layer must allow to stream data in form of frames (fixed size or not)
  2. Simple but reliable, unambiguous packet framing protocol, frame should be easy to identify
  3. Fast frame synchronization: When an endpoint (Arduino or ROS) connects to the Serial link in the middle of the data streaming, frame synchronization should be fast while minimizing the frames lost in the synchronization phase
  4. Frame should be verified using checksum before being consumed by an endpoint
  5. Packet framing overhead is allowed but need to be minimized
  6. The algorithms should be easy to implement and computationally inexpensive on both Jetson and Arduino

Brief, we need an efficient and reliable delimiting/synchronization scheme to detect the frame with short recovery time.

The detail on the choice of protocol and algorithm as well as insight implementation is presented on a section of my Jarvis booklet accessible via the following link:

https://doc.iohub.dev/jarvis/Ym9vazovLy9jXzIvc18xL0lOVFJPLm1k/JETTY:_Jarvis_Serial_to_ROS-2_transport_layer.md

Control GPIO using the new Linux user space GPIO API

From the version 4.8, the Linux kernel introduces a new user space API based on character devices for managing and controlling GPIOs ( General-Purpose Input/Output). This post presents the basic of the new interface as well as a simple tutorial/example to demonstrate how to use the new API to control GPIOs.

The hardware used in the tutorial is the Raspberry Pi 3B but the code is generic and can be used on any embedded hardware.

26/06/2020 docker, AntOS, image, container, API

Running your own AntOS VDE system using docker image

[Update 27/03/2021]: The image is now available on docker hub at: https://hub.docker.com/r/xsangle/antosaio (image: xsangle/antosaio:latest). This post has been updated with latest instruction on how to host a local instance of Antos VDE using docker.

More information on AntOS https://doc.iohub.dev/antos

The docker image provide out-of-the-box working AntOS system which consist of:

  • The web-server
  • AntOS server side API
  • AntOS client side API

AntOS user can be created at creation of of the container, each user is attached to a separated container.

In a multi-users scenario, This mean that user working instance is completely isolated to other. Resource consumption such as CPU, storage and RAM of each instance is configurable.

How to list supported ciphers suites of a server?

I run into a problem of how to check whether my SSL ciphers suites configuration works correctly on my server.
Basically, with openssl, client can verify if the server supports a particular cipher suite using the following command:

openssl s_client -cipher "$cipher" -CAfile ca/ca.crt -connect server:port
# $cipher is the cipher suite name

So it is possible to automatically test all cipher suites supported by openssl against the server using a simple snippet of Bash, i found such script in this site https://superuser.com/questions/109213/how-do-i-list-the-ssl-tls-cipher-suites-a-particular-website-offers and modify it a little bit. Below is the script:

#!/usr/bin/env bash

# OpenSSL requires the port number.
SERVER=$1
DELAY=1
ciphers=$(openssl ciphers 'ALL:eNULL' | sed -e 's/:/ /g')

echo Obtaining cipher list from $(openssl version).

for cipher in ${ciphers[@]}
do
    echo -n Testing $cipher...
    result=$(echo -n | openssl s_client -cipher "$cipher"  -connect $SERVER 2>&1)
    if [[ "$result" =~ ":error:" ]] ; then
        error=$(echo -n $result | cut -d':' -f6)
        echo NO \($error\)
    else
        if echo $result | grep -q "Verify return code: 0 (ok)"; then
            echo YES
        else
            echo UNKNOWN RESPONSE
            echo $result
         fi
    fi
sleep $DELAY
done

AntOS 1.0.0-alpha

Github: https://github.com/lxsang/antos branch antos-1.0.0a

Demo: https://app.iohub.dev/antos/ using user name and password: demo/demo

If one wants to run AntOS VDE locally in their system, a docker image is available at:
https://github.com/lxsang/antosaio

API Documentation: https://doc.iohub.dev/antos

It has been a long time since version 0.x.x and now AntOS hits a major changes in its API. From version 1.0.0, AntOS no longer depends on Riot.js in its core UI API. This version introduces a brand new AntOS UI API called AFX API which is rewritten from bottom up. The entire AntOS core API is rewritten in Typescript (from Coffeescript) for better debugging, code maintenance and documenting.

Browser support: tested on Chrome, Firefox and partly Safari. Any browser that supports custom elements API should work. May have problem with Microsoft Edge.

Rust tip: (Unix) drop the current user privileges

Brief

Rust is a modern programing language which is claimed to be blazingly fast and memory-efficient. It syntactically similar to C++, but is designed to provide better memory safety while maintaining high performance and productivity:

  • Zero cost abstraction: allow a perfect balance between performance and productivity
  • Memory efficient with no runtime or garbage collector
  • Memory safe: Rust does not permit null pointers, dangling pointers, or data races in safe code.
  • Memory management using an ownership model guarantee memory-safety and thread-safety .
  • Great documentation, easy to use compiler and integrated packages/libraries management
  • Easy to interface with other language.
  • A bit of learning curve for the variable ownership and variable lifetime features.
Powered by antd server, (c) 2017 - 2021 Xuan Sang LE