This Case Shows Specific Expertise. Find the Companies with the Skills Your Project Demands!

You're viewing one of tens of thousands of real cases compiled on Many.dev. Each case demonstrates specific, tangible expertise.

But how do you find the company that possesses the exact skills and experience needed for your project? Forget generic filters!

Our unique AI system allows you to describe your project in your own words and instantly get a list of companies that have already successfully applied that precise expertise in similar projects.

Create a free account to unlock powerful AI-powered search and connect with companies whose expertise directly matches your project's requirements.

Intel Gaudi Optimization for LLM Deployment

vstorm.co

Information technology

Financial services

Healthcare

Challenge: Hardware Specialization and LLM Portability

Global AI Solutions Inc. is experiencing limitations in deploying Large Language Models (LLMs) due to reliance on NVIDIA hardware. They need to support LLM deployments on Intel Gaudi AI accelerators to broaden hardware options, reduce vendor lock-in, and optimize performance for specific workloads. The existing LLM codebase, primarily optimized for NVIDIA's CUDA platform, requires significant adaptation for the Intel Gaudi architecture.

About the Client

A multinational technology company specializing in developing and deploying AI solutions for enterprise clients.

Project Goals

Successfully port the Llama model to run efficiently on Intel Gaudi hardware.
Develop a robust backend support for the ggml library on the Intel Gaudi platform.
Achieve comparable or improved performance on Intel Gaudi compared to NVIDIA GPUs for targeted LLM workloads.
Ensure seamless integration of the ported Llama model with existing AI infrastructure and workflows.
Create a reusable framework for porting other LLMs and AI models to the Intel Gaudi architecture.

Functional Requirements

Llama model execution on Intel Gaudi accelerators.
Support for the ggml library with optimized kernels for Intel Gaudi.
Data transfer and memory management optimized for Gaudi.
Performance monitoring and profiling tools for Gaudi-based Llama deployments.
Integration with existing AI infrastructure for model loading and deployment.

Preferred Technologies

Intel Gaudi AI accelerators (HPU)

ggml library

C/C++

TCPC kernel code for Gaudi 3 device

Glue code for Host (CPU)

Integrations Required

Existing AI model management platform
Data storage and retrieval systems
Monitoring and logging tools

Non-Functional Requirements

Scalability to support varying model sizes and workloads.
High performance and low latency for LLM inference.
Reliability and stability of the Gaudi-based deployment.
Security to protect sensitive data and model assets.
Maintainability and extensibility for future model updates and enhancements.

Expected Business Impact

Successful completion of this project will enable Global AI Solutions Inc. to offer LLM deployment on a wider range of hardware, reduce reliance on a single vendor, and optimize performance for specific customer needs. This will enhance their competitive advantage in the AI solutions market and open up new business opportunities. It will also provide clients with more cost-effective and flexible AI infrastructure options.