BirdCLEF 2026 Starter Case Study

A knowledge distillation case study using EfficientNetB0 to identify overlapping bird calls in noisy field recordings from the Pantanal, mimicking Google’s Perch v2 with multi-task learning.

Quick Facts

Context: Kaggle Competition (BirdCLEF 2026), Independent Case Study
Tech Stack: Python, TensorFlow, Keras, EfficientNetB0, Google Colab, Librosa, Google Perch v2
Collaborators: Engin Deniz Dogu, Gabriel Reynoso Romero
Links: Kaggle Notebook | LinkedIn Post

Overview and Problem

The BirdCLEF 2026 competition challenges participants to identify overlapping bird calls in noisy and continuous field recordings from the Pantanal biome. This is a hard bioacoustics problem because recordings are long, multi-species calls overlap, and real-world field noise is substantial. Although we joined late and treated this as a learning exercise rather than a competitive submission, we aimed to understand and document the winning approaches from prior editions.

What I Built

Architected a multi-task student model (EfficientNetB0) that simultaneously learns from both soft labels generated by Google’s Perch v2 teacher model and hard labels from human-annotated soundscapes.
Implemented knowledge distillation to transfer Perch v2’s rich bioacoustic representations into a compact, trainable model — heavily inspired by Nikita Babych’s winning approach.
Documented the full pipeline end-to-end with explanations and comments, from raw audio loading and mel-spectrogram extraction to model training and inference, creating a readable reference notebook for the community.

Architecture overview

flowchart LR
    AUD[Field audio] --> MEL[Mel-spectrogram]
    MEL --> STU[EfficientNetB0 student]
    AUD --> PERCH[Perch v2 teacher]
    PERCH -->|soft labels| LOSS[Multi-task loss]
    HUM[Human soundscape annotations] -->|hard labels| LOSS
    STU --> LOSS
    LOSS --> STU

Example model input and target

Chunk 0 of a field recording: the mel spectrogram fed to the student (left) and the 1536-D Perch v2 embedding it learns to reproduce, reshaped into a grid (right).

Key Results and Impact

Achieved a private leaderboard score of ~0.60 after only 3 training epochs under severe compute constraints (Google Colab timeouts).
Demonstrated that multi-task distillation (soft labels from Perch + hard labels from human annotations) acts as a powerful regularizer, preventing overfitting to the teacher’s biases — as noted by community feedback.
Published a well-commented starter notebook that bridges the gap between beginner and competitive-level solutions in bioacoustic ML.

Core Learnings

Knowledge distillation with dual supervision — combining soft (teacher) and hard (human) labels is a principled way to regularize and improve generalization on noisy audio tasks.
Trade-offs between model capacity and compute budget — EfficientNetB0 strikes the right balance for limited GPU/TPU environments; larger models would require more epochs to converge.
Bioacoustic-specific preprocessing matters — mel-spectrogram parameterization (hop length, n_mels, frequency range) heavily influences what spectral patterns the model can learn from bird calls.
Perch v2 as a foundation model for audio — Google’s Perch model provides strong general-purpose audio embeddings that transfer well to ecological monitoring tasks.

Useful Resources

History of BirdCLEF Competitions in Kaggle — A detailed overview of how the BirdCLEF competition has evolved over the years on Kaggle.
BirdCLEF Audio Identification Tutorial Video — A video guide illustrating key strategies and insights for BirdCLEF bioacoustic classification.

LinkedIn Post

We were a little late to this year’s BirdCLEF 2026 competition, but Gabriel Reynoso Romero and I wanted to use this as a case study and learn about previous submissions. We created a notebook with explanations and comments that go from beginning to end.

The challenge is about bioacoustics, specifically, identifying overlapping bird calls in noisy and continuous field recordings in the Pantanal. We built a multi-task student model (EfficientNetB0) that mimics Google’s Perch v2 while simultaneously learning from human-annotated soundscapes, heavily inspired by Nikita Babych’s winning approach. Due to resource limits (and Colab kept giving up on us), we only trained it for 3 epochs, but the distillation strategy still managed to hit a private score of ~0.60.

Source: LinkedIn — Engin Deniz Dogu

Explorer