Linux Developer Conference Brazil 2019 speaker: Eduardo Silva
Eduardo is a Principal Engineer at ARM / Treasure Data. He currently leads the efforts to make logging and data processing more friendly and scalable in Embedded and Containerized systems such as Kubernetes. Maintainer of Fluent Bit, a Fluentd open source sub-project.
Logging & Stream Processing on the Edge (in English)
In IoT, Data Analysis is fundamental to gather insight from applications and business in general and Logging one of the key components to collect and pre-process data, usually, a logging mechanism goes through the normal workflow of collect, parse, filter and centralize logs to a storage backend like a database, so data processing and analysis can be performed.
Data Processing usually happens after the data has been aggregated and stored, but for real-time analysis needs, process the data while is still in motion brings a lot of advantages and this kind of approach is called Stream Processing.
In this presentation, we will go further and present an extended approach called 'Stream Processing on the Edge', where data is processed on the edge service or device, in a lightweight mode empowering features like anomaly detection and Machine Learning using Fluent Bit.