Organisation Profile
KORNIA
Kornia is a high-performance, differentiable computer vision library for PyTorch. Beyond classic operators, it provides a production-ready ecosystem for State-of-the-Art (SOTA) Vision-Language Models (VLM) and Vision-Language-Action (VLA) models, enabling seamless integration of perception and reasoning for the next generation of AI agents.
MENTORS
PROJECTS
High-Performance Augmentation Benchmarking
Problem Statement
Kornia is architected for speed, but we currently lack a robust, automated method to track performance regressions over time. We need to rigorously benchmark our augmentation pipeline against competing libraries (like Albumentations or torchvision) to ensure we maintain our competitive edge.
Focus Area
Performance Optimization, DevOps/CI, Data Visualization.
Student Contribution Guide
• Optimize Performance: Refine kornia.augmentation logic to achieve the highest possible speed. • Create Benchmarks: Develop a standardized benchmarking suite specifically for Kornia augmentations. • Automate Tracking: Integrate these benchmarks into our CI/CD pipelines to automatically detect performance drops in new Pull Requests. • Analyze Throughput: Conduct comprehensive comparisons of throughput (images/second) across different hardware (CPU vs. GPU) and modes (single image vs. batch).
Data and Fine-Tuning API + Model Expansion for VLMs
Problem Statement
Vision-Language Models (VLMs) represent the current frontier of AI. To stay relevant, Kornia requires a streamlined, native API to fine-tune these models on niche datasets. Additionally, the library needs to be expanded to support the latest VLM architectures.
Focus Area
Backend API Development, Model Modeling, Deep Learning.
Student Contribution Guide
• Build the API: Implement an experimental, 'Kornia-native' kornia.data and fine-tuning API (kornia.train) designed specifically for VLMs. • Expand Model Support: Integrate 2-3 modern VLM architectures into the Kornia ecosystem and validate them via fine-tuning. • Demonstrate Usage: Create high-quality example notebooks for downstream tasks such as Visual Question Answering (VQA) or Image Captioning.
Universal Compatibility: ONNX Export & Torch.compile
Problem Statement
For production deployment, Kornia operators must be easily exportable. This project ensures the entire codebase is compatible with torch.compile (full graph mode) and exports cleanly to ONNX without relying on inefficient 'fallback' operations.
Focus Area
Production Engineering, Graph Compilers, Performance Optimization.
Student Contribution Guide
• Audit Codebase: Review current modules to identify compatibility gaps with torch.compile. • Refactor Logic: Rewrite non-traceable code (such as dynamic branching) to be symbolic-friendly. • Verify Outputs: Establish a strict verification suite to ensure ONNX exports match PyTorch outputs within a precise epsilon.
Robot Learning (Advanced)
Problem Statement
We aim to extend Kornia's capabilities into the robotics domain, focusing on differentiable rendering, spatial transformations, and policy learning. This requires high-level qualifications in robotics integration. Team must have a robot (with mobile platform)
Focus Area
Robotics, Imitation Learning.
Student Contribution Guide
• Integrate Platform: expand bubbaloop to connect a specific robot (hardware, depending on availability) to a new robot learning framework, create datasets, and train a VLA. https://github.com/kornia/bubbaloop • Structure Data: Iterate on and improve the data structure used for robot learning datasets to ensure scalability and ease of use.
Ready to collaborate?
Join the community chat, review the issue tracker, and pick a project to start contributing. Mentors are available to help you scope your first patch.
