AI Research Engineer – Kernel & Inference Optimization
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI Research Engineer focusing on model serving and inference architectures at Tether. Join the fintech revolution with cutting-edge AI systems and optimizations.
Responsibilities:
- Design and deploy state-of-the-art model serving architectures that deliver high throughput and low latency while optimizing memory usage.
- Ensure these pipelines run efficiently across diverse environments, including resource-constrained devices and edge platforms.
- Establish clear performance targets such as reduced latency, improved token response, and minimized memory footprint.
- Build, run, and monitor controlled inference tests in both simulated and live production environments.
- Track key performance indicators such as response latency, throughput, memory consumption, and error rates, with special attention to metrics specific to resource-constrained devices.
- Document iterative results and compare outcomes against established benchmarks to validate performance across platforms.
- Identify and prepare high-quality test datasets and simulation scenarios tailored to real-world deployment challenges, specifically those encountered on low-resource devices.
- Set measurable criteria to ensure that these resources effectively evaluate model performance, latency, and memory utilization under various operational conditions.
- Analyze computational efficiency and diagnose bottlenecks in the serving pipeline by monitoring both processing and memory metrics.
- Address issues such as suboptimal batch processing, network delays, and high memory usage to optimize the serving infrastructure for scalability and reliability on resource-constrained systems.
- Work closely with cross-functional teams to integrate optimized serving and inference frameworks into production pipelines designed for edge and on-device applications.
- Define clear success metrics such as improved real-world performance, low error rates, robust scalability, optimal memory usage and ensure continuous monitoring and iterative refinements for sustained improvements.
Requirements:
- A degree in Computer Science or related field.
- Ideally PhD in NLP, Machine Learning, or a related field, complemented by a solid track record in AI R&D (with good publications in A* conferences).
- Must have knowledge of Metal Shading Language (MSL).
- Proven experience in low-level kernel optimizations and inference optimization on mobile devices is essential.
- Your contributions should have led to measurable improvements in inference latency, throughput, and memory footprint for domain-specific applications, particularly on resource-constrained devices and edge platforms.
- A deep understanding of modern model serving architectures and inference optimization techniques is required.
- Must have strong expertise in writing GPU kernels for mobile devices (i.e., smartphones) as well as a deep understanding of model serving frameworks and engines.
- Practical experience in developing and deploying end-to-end inference pipelines, from optimizing models for efficient serving to integrating these solutions on resource-constrained devices is required.
- Demonstrated ability to apply empirical research to overcome challenges in model serving, such as latency optimization, computational bottlenecks, and memory constraints.
- You should be proficient in designing robust evaluation frameworks and iterating on optimization strategies to continuously push the boundaries of inference performance and system efficiency.
- Distributed Inference Systems: Designing and optimizing high-performance inference engines using techniques like Tensor Parallelism, Pipeline Parallelism, and Expert Parallelism to handle massive models on GPU clusters.
- Deep understanding of the math and structure behind Diffusion Models and Vision Transformers.
- Understanding of Pruning, Quantization, Flash attention, KV Cache, Speculative Decoding (Eagle) etc.
Benefits:
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
Similar Jobs
AI Research Engineer, Kernel & Inference Optimization
AI Research Engineer driving innovation in model serving and inference architectures for advanced AI systems at Tether. Focusing on optimizing models for responsive, efficient, and scalable AI performance.
AI Research Engineer – Kernel, Inference Optimization
AI Research Engineer at Tether focusing on optimizing model serving and inference architectures for advanced AI systems. Designing scalable performance solutions affecting real-world applications.
AI Research Engineer – Kernel & Inference Optimization
AI Research Engineer at Tether focusing on optimizing model serving and inference architectures. Collaborate globally to develop cutting-edge fintech solutions.
AI Research Engineer – Kernel & Inference Optimization
AI Research Engineer responsible for optimizing model serving and inference architectures. Join Tether to innovate in the fintech space remotely from India.
AI Research Engineer – Kernel & Inference Optimization
AI Research Engineer focusing on model serving and inference at Tether, contributing to advancements in AI systems and architecture. Collaborating with a global team in a dynamic fintech environment.
Legal Expert – AI Research Insights
Participate in a research study collecting real-world legal scenarios for AI training. Seeking legal professionals across various qualifications to aid in AI model training.
AI Research Engineer
AI Research Engineer designing and developing next-generation agentic AI systems. Leading the transition of cutting-edge research into scalable solutions for the cybersecurity field.
Intern – Agentic AI Research
Intern supporting research and development of advanced healthcare informatics solutions with Agentic AI for Cotiviti. Collaborating with cross-functional teams for practical implementation in healthcare environments.
AI Research Engineer
AI Research Engineer conducting independent ML research and contributing to the team's core assets. Involves designing complex experiments and contributing high-quality code for the organization's projects.
AI Research Engineer, Computer Vision
AI Research Engineer in a social AI company focusing on image and video generation. Collaborating with researchers to implement cutting-edge machine learning models.
Staff AI Scientist – AI Products, Life Sciences
Staff AI Scientist at Tempus developing AI-driven precision medicine solutions. Leading AI projects and designing multimodal AI models to improve clinical care.
AI Research Intern, Virtual Cell Model
AI Research Intern developing next generation AI/ML models for virtual cell modeling. Join J&J's Data Science R&D team and contribute to cutting-edge biomedical research.
Insurance Professional – AI Residency, Part-time
Insurance Professionals developing AI models in finance and insurance sectors. Seeking senior analysts and portfolio managers for research roles to push limits of AI reasoning.
Commercial Banking Professional – AI Residency, Part-time
Commercial Banking Professionals working with AI to reshape banking workflows in corporate finance. Ideal for experienced analysts and managers in credit and risk sectors.
Staff Machine Learning Engineer, AI Researcher
Staff Machine Learning Engineer at Cribl shaping AI-enabled security platforms. Collaborating with experts to integrate cutting-edge AI technologies for real customer solutions.
Chief ML Researcher, Product
Chief ML Researcher leading machine learning initiatives for Nebius, shaping product direction and platform capabilities. Engaging strategic customers and driving forward-looking ML research to innovate AI solutions.
Real Estate Investment Professional – AI Residency, Part-time
Real Estate Investment Professionals in AI Residency at Mentis AI, focusing on how AI impacts investment workflows.
Real Estate Investment Professionals – AI Residency, Part-time
Real Estate Investment Professionals utilizing AI to enhance investment workflows in private equity. Engaging in benchmarking, model testing, and framework design for AI systems.
Private Credit Professional – AI Residency, Part-time
Private Credit Professional working with AI systems in a flexible, remote environment. Developed for experts in private credit and advanced AI applications.
Private Credit Professional – AI Residency, Part-time
Private Credit Professionals working on AI systems for credit workflows in finance. Evaluating and comparing human judgment with AI outputs for middle-market lending.