|
| Reduce CPU spikes - AI Summarization |
Summarization aims to compress a lengthy source document into a concise format while retaining its core components and key ideas.
However, when you are hosting your own LLM, handling CPU spikes (in the absence of a GPU) can be your biggest concern.
 |
| GPU workloads on k3s |
K3s is a highly available, certified Kubernetes distribution designed for production workloads. It can also be used for AI workloads.
By default, k3s nodes do not recognize GPUs. In this article, we will enable k3s to work with a GPU.
 |
| EmbeddingGemma on NVIDIA Triton Server |
 |
| AI Crawl Control From Cloudflare |
Cloudflare has recently announced a new feature called AI Crawl Control .
A performance test is extremely important for an Artificial Intelligence (A.I.) Model just as it is important for an e-commerce website like Amazon.
 |
| NVIDIA Triton Inference Server |
What is an Inference Server?
The role of an Inference Server is to accept user input data and pass it to an underlying trained model in the required format and return the results. It is also widely known as a Prediction Server as the results are nothing but predictions (in most cases).
The NVIDIA Triton server is a gold standard that standardizes AI model deployment and execution across every workload and it is important to know how it works internally for your custom or off the shelf models.
 |
| Ollama.ai |
Ollama.ai is an excellent tool that helps you to run Large Language Models (LLMs) locally on your computer like Llama2.
In this article, I decided to test whether Ollama can work with my consumer grade GPU - MSI GTX Super 1660
