You can now chat with your GPU
NVIDIA enables new AI features for RTX 30/40 users.
NVIDIA has introduced a new feature called “Chat with RTX,” offering an alternative for users uncomfortable with online AI chats. This AI chatbot operates locally on GeForce RTX 30 and 40 GPUs, leveraging TensorRT-LLM & Retrieval Augmented Generated (RAG) software fully supported by RTX GPUs and accelerated by built-in Tensor cores.
NVIDIA’s Chat with RTX is all about letting everyone tap into the power of their graphics cards for AI tasks. We’ve mostly seen AI doing cool stuff with images and videos, but until now, chats were kind of stuck in big data centers because of the resources needed for those complex AI models with tons of variables.
Now, users can download a lighter version of these AI models, some of which available for download for free. But, like with any new tech, getting it set up for regular users can be a bit tricky. NVIDIA apparently wanted to make this process a lot simpler through a single app that runs on your machine, and you can fire it up through your browser, making the whole process way more user-friendly.
Chat with RTX is more than a mere text processor, it can access various file formats, including text, pdf, doc/docx, and xml. It harnesses the capabilities of popular Large Language Models like Mistral or Llama2 to generate responses and can also utilize online resources such as YouTube videos.
Currently, NVIDIA confirms support only for GeForce RTX 30 and RTX 40 GPUs, without mention of the RTX 20 series. The GPUs must have a minimum of 8GB of VRAM, excluding the RTX 3050 6GB from compatibility.
The chat will be free for download soon, you can check the following link for more information.
Source: NVIDIA via Wccftech