🤗 Kernels: Major Updates

In our previous deep dive, "From Zero to GPU," we introduced the 🤗 Kernels project—a bold initiative designed to standardize the packaging, distribution, and consumption of custom kernels. Our mission has always been to ensure this process remains frictionless and secure while fostering deep integration with the Hugging Face Hub. Over the past few months, we have undergone a comprehensive redesign of the project. This article outlines the major milestones we have achieved and provides a roadmap for what lies ahead.

Kernels: A New Repository Type

We have officially introduced a dedicated repository type on the Hub specifically for "kernels." This evolution allows us to provide a tailored experience for users dealing with complex compute-related requirements. By categorizing kernels as first-class citizens, users can now easily identify which accelerators, operating systems, and backend versions are supported for any given kernel.

"Making these kernels first-class citizens of the Hub benefits the entire AI ecosystem. Users can now observe trends across kernels, models, and the applications that leverage them, significantly increasing discoverability."

You can browse the full catalog of available kernels on the Hub at huggingface.co/kernels.

Elevating Security Standards

Because kernels execute native code with the same privilege level as the Python process that loads them, the potential for malicious activity is significant. Security has been the cornerstone of our development. We have prioritized reproducibility, ensuring that developers can recompile kernels themselves to verify they match the publicly available source code.

To achieve this, we utilize Nix, which facilitates pure builds through hermetic evaluation and a strongly isolated sandbox. Furthermore, we enhance provenance by embedding the source Git SHA1 directly into the kernel binary. Recently, we have introduced two critical layers of defense: Trusted Kernel Publishers and Code Signing.

Trusted Kernel Publishers

To mitigate the risk of users inadvertently running malicious code, the kernels package now defaults to loading kernels only from "trusted publishers." These are organizations vetted by the community for their commitment to security and good faith.

If you wish to load a kernel from an unverified source, you must explicitly opt-in using the trust_remote_code argument:

from kernels import get_kernel

kernel_module = get_kernel(
    "Atlas-Inference/gdn", 
    version=1, 
    trust_remote_code=True
)

Publishing kernel repositories is restricted by default. Organizations and individual developers must request access via their account settings, allowing us to review each request on a case-by-case basis.

Kernel Signing

Code signing provides a safeguard against scenarios where a trusted publisher’s credentials might be compromised. By signing a kernel with a private key, we ensure that even if an attacker gains access to a repository, they cannot sign a malicious payload without the original developer's private key.

We have integrated Sigstore’s cosign to facilitate signing with ephemeral keys, which are valid only for a limited duration. Additionally, we verify that the kernel was signed by a trusted GitHub workflow. While kernels currently supports signature verification via the kernels verify-signature command, we are testing this functionality before making it a mandatory requirement for loading.

Revamped CLI Experience

Previously, the utilities for kernels and kernel-builder were somewhat intertwined. We have now established a clear separation of concerns:

• kernels: A library focused on loading and preparing kernels for execution.
• kernel-builder: A dedicated tool for the creation and compilation of kernels.

By decoupling these, both tools have become leaner and more specialized. This architectural shift also better supports the emerging field of agentic kernel development.

Expanded Framework and Backend Support

We have significantly broadened our support for various frameworks and backends to ensure maximum flexibility for developers:

• Torch Stable ABI: We now support the Torch Stable ABI, allowing developers to target a specific Torch version and maintain compatibility for approximately two years. A kernel targeting the Torch 2.9 Stable ABI, for example, will function with any version released thereafter.
• Apache TVM FFI: This is the first framework supported beyond Torch. The TVM FFI provides a standardized ABI that enables interoperability across PyTorch, Jax, and CuPy, allowing developers to write kernels that run seamlessly across different ecosystems.

The Foundation for Agentic Kernel Development

The combination of kernel-builder and kernels provides a robust foundation for agentic workflows, where AI agents are tasked with writing, benchmarking, and optimizing kernels from scratch.

Agentic development is still in its infancy, but our tools provide the necessary structure to make these workflows repeatable and predictable. kernel-builder enforces a project layout that agents can easily navigate, while its CLI is designed for non-interactive, programmatic interpretation.

Benchmarking and Optimization

Building a kernel is merely the first step. To ensure actual performance gains, we have integrated with HF Jobs. This allows agents to: 1. Execute benchmark suites across various hardware configurations. 2. Collect performance metrics. 3. Compare results against a defined baseline.

This feedback loop is essential for agents to iterate on their optimizations, ensuring that the final kernel delivers tangible speedups on the target hardware. You can view examples of agent-augmented kernels at drbh/yamoe and sayakpaul/qk-norm-rope.

Miscellaneous Improvements

Streamlined Environment Setup

We recognize that setting up a build environment can be complex. We have introduced a one-click installation script to simplify the process. For those working in cloud environments, our new Terraform setup guide provides a streamlined path to deploying ephemeral build instances.

System Cards

Every kernel now features a "System Card," which acts as the front matter for the repository on the Hub. This card exposes critical information, including usage instructions and interface definitions, ensuring that users have all the context they need before integrating a kernel into their stack.

Compatibility Checks

Determining if a kernel is compatible with your system is now easier than ever. You can use the has_kernel() method for a quick boolean check, or get_kernel_variants() for a detailed breakdown of why a specific variant might be rejected:

from kernels import get_kernel_variants, VariantAccepted

for decision in get_kernel_variants("kernels-community/activation", version=1):
    name = decision.variant.variant_str
    if isinstance(decision, VariantAccepted):
        print(f"{name}: compatible")
    else:
        print(f"{name}: rejected ({decision.reason})")

Improved manylinux_2_28 Support

We have addressed previous issues regarding libstdc++ compatibility. By moving away from static linking—which caused data corruption due to global initialization conflicts—we now link libstdc++ dynamically. To maintain compatibility with older systems, we have standardized our compilation process using the official manylinux_2_28 toolchain.

Conclusion

Our goal with the 🤗 Kernels project remains steadfast: to provide a world-class experience for both kernel developers and the users who rely on their high-performance code. These updates represent a significant leap forward in security, usability, and agentic readiness.

We are constantly iterating based on community feedback, and we encourage you to contribute to the project. Whether you are building the next generation of optimized kernels or integrating them into your research, your input is invaluable.

*

Acknowledgements: Special thanks to Aritra for his thorough review of this post.