Map of Contents
Notes
- CRISP-DM -
Source: [CRISP-DM: das Standard-Vorgehensmodell für Data Mining](https…
Statistics
- Correlation Coefficients - Pearson, Spearman, and Kendall, when each fits, and why zero correlation is not independence. Related: Exploratory Data Analysis.
- Statistical Inference Basics - p-values, confidence intervals, t-tests with worked examples, business decisions, and Bayesian credible intervals. Related: 1. Probability.
Big Data Technologies
Read in this order: foundations, storage and batch compute, then streaming, plumbing, and analytics.
- Big Data Fundamentals and Architecture - The four Vs, layered architecture, storage options, CAP trade-offs, and responsible use. Related: Database Types, Database Selection.
- Hadoop and MapReduce - HDFS, YARN, the map-shuffle-reduce model, and the Hadoop ecosystem.
- Apache Spark - In-memory engine: RDDs, DataFrames, lineage, and how it differs from MapReduce.
- Stream Processing and Event Streaming - Batch vs. stream, Kafka, Storm, Flink, Spark Streaming, and cloud services.
- Big Data Integration and Orchestration - NiFi, Kafka, Flume, Airflow, and Oozie for ingestion and scheduling.
- Distributed Machine Learning and Graph Processing - Mahout, MLlib, Giraph, GraphX, and Neo4j.
- Correlation Drift Monitoring with Spark - Worked predictive-maintenance example: batch correlation matrix, sliding-window streaming correlation, and alerting pitfalls.
- Ray - Python-first task and actor runtime for distributed ML, tuning, RL, and serving; compared with Spark.
External Resources & Data
- Wikimedia Commons - Massive open-source media repository; useful for visual datasets and public domain assets.