Overview
This is a memo on tools related to LLMs.
LangChain
It is described as follows.
LangChain is a composable framework to build with LLMs. LangGraph is the orchestration framework for controllable agentic workflows.
LlamaIndex
https://docs.llamaindex.ai/en/stable/
It is described as follows.
LlamaIndex is a framework for building context-augmented generative AI applications with LLMs including agents and workflows.
LangChain and LlamaIndex
The response from gpt-4o was as follows.
Both LangChain and LlamaIndex are frameworks that support application development using LLMs (Large Language Models).
From a brief investigation, it appeared that LlamaIndex is easier to use when performing RAG (Retrieval-Augmented Generation).
Ollama
https://github.com/ollama/ollama
It is described as follows.
Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models.
It appears to be a tool for running LLMs in a local environment.
Download from the following page.
After launching the app, it could be used from commands like the following.
Additionally, by combining with LlamaIndex, Ollama could also be used from Python.
https://docs.llamaindex.ai/en/stable/api_reference/llms/ollama/
OpenRouter
https://openrouter.ai/docs/quick-start
It is described as follows.
OpenRouter provides an OpenAI-compatible completion API to 278 models & providers that you can call directly, or using the OpenAI SDK. Additionally, some third-party SDKs are available.
After account registration, you can select a model and try chat and other features.
By purchasing credits, gpt-4o and other models could also be used.
Additionally, by combining with LlamaIndex, OpenRouter could also be used from Python.
https://docs.llamaindex.ai/en/stable/api_reference/llms/openrouter/
After issuing an API key in OpenRouter, the following script could be executed.
The following is an example using the model Meta: Llama 3.2 1B Instruct (free).
The following is an example using the model OpenAI: GPT-4o (2024-11-20). By only changing model, the same script could be used.
Model names can be found on pages like the following.
https://openrouter.ai/meta-llama/llama-3.2-1b-instruct:free
LiteLLM
It is described as follows.
LLM Gateway to manage authentication, loadbalancing, and spend tracking across 100+ LLMs. All in the OpenAI format.
The following describes how to use OpenRouter from LiteLLM.
https://docs.litellm.ai/docs/providers/openrouter
It could be executed with the following code.
Summary
I have not fully understood the differences between the tools mentioned here, and there are likely many other useful tools available, but we hope this article serves as a useful reference for understanding the tools surrounding LLMs.