Tag: Data Processing

How To Install Apache Spark on Ubuntu

Posted on September 16, 2020 by Justin Palmer | Updated: March 8, 2021

Category: Tutorials | Tags: Apache Spark, Big Data, Cluster Computing, Clustering, Clusters, Data Driven Applications, Data Processing, Data Science, Distributed Systems, Environment Variables, Framework, HA Cluster, Hadoop, Java, local, Machine Learning, Master, Pyspark, Scala, Shell, Spark, Spark Shell, Web Interface, Worker

Reading Time: 6 minutes

What is Apache Spark?

Apache Spark is a distributed open-source, general-purpose framework for clustered computing. It is designed with computational speed in mind, from machine learning to stream processing to complex SQL queries. It can easily process and distribute work on large datasets across multiple computers.

Continue reading →

How to Install Logstash

Posted on August 17, 2020 by Justin Palmer | Updated: March 8, 2021

Category: Tutorials | Tags: Apt Transport HTTPS, Data Filters, Data Pipeline, Data Processing, ElasticSearch, ELK, Event Pipeline, Java, Java JDK, Kibana, Log Collector, Log Collectors, Log Files, Logs, Logstash, Ubuntu, Ubuntu 18.04

Reading Time: 5 minutes

What is Logstash?

Logstash is a free, open source, server-side data collection and processing engine which uses dynamic pipelining capabilities. It takes in information from multiple data sources, reconstructs it, and then sends it to a destination of our choice. Logstash also cleans and modifies the data for use in an advanced outbound analytics and visualization use cases. It is the L in the ELK stack (Elasticsearch, Logstash and Kibana) and is typically responsible for sending data to Elasticsearch.

Continue reading →