Run LLMs Locally with Ollama

-

Ollama is used to run and manage large language models (LLMs) locally on your own computer for offline, private, and efficient AI operations. It simplifies downloading, running, and customizing open-source LLMs like Llama 3 or Mistral through a simple command-line interface, making it a valuable tool for developers, researchers, and anyone concerned with data privacy.

Ollama server is used to serve large language models (LLMs) locally on a user's machine, making them accessible via an API, a web UI, or through various applications like code assistants. It provides a simple way to download, run, and manage models for tasks such as building chatbots, developing applications, and conducting research, with a focus on local execution for data privacy and lower latency.











Here are simple steps to work on this exercise:

1) Install Ollama: Download and install Ollama from Ollama Download

2) Start the server: Run ollama serve in a separate terminal to make the server available. (This is often automatic on startup, but it's good to verify.)

Eg: C:\Users\Admin\AppData\Local\Programs\Ollama>ollama serve

Article content

Check the browser to track the Ollama server status:


Article content

3) Pull a model: Download a model to your local machine.

Eg: C:\Users\Admin\AppData\Local\Programs\Ollama>ollama pull llama3.2

Article content


Few of the below options I tried to get the response from a simple Prompt: What is Large Language Model?

Option#1: Run the below command from command line and enter the prompt in

>>> Send a message (/? for help) and the response comes from the selected llma3.2 model like below.

C:\Users\Admin\AppData\Local\Programs\Ollama>ollama run llama3.2

Article content

Option#2: Use curl command to use Ollama API service

C:\Users\Admin\AppData\Local\Programs\Ollama>curl http://localhost:11434/api/generate -d "{\"model\": \"llama3.2\", \"prompt\": \"What is a large language model?\",\"stream\": false}"

JSON response containing the model's answer:

Article content

Option#3: Run the Ollama API from Postman but for this we need to install the Desktop app since the Ollama server runs on localhost

Article content

Option#4: Run a python program & the package: langchain, langchain-ollama, ollama with the below steps using Windows

1) Install Python3 and above

2) Open the Visual Studio env to run the below DOS commands from the terminal inside Visual Studio

1) C:\Users\Admin>python -m venv chatbot

2) (chatbot) C:\Users\Admin\>cd chatbot\Scripts

3) (chatbot) C:\Users\Admin\chatbot\Scripts>activate.bat

4) (chatbot) C:\Users\Admin\chatbot\Scripts>pip install langchain langchain-ollama ollama

  • langchain: This installs the core LangChain framework.

  • langchain-ollama: This installs the specific integration package that allows LangChain to interact with Ollama.

3) Add a new main.py file under C:\Users\Admin\chatbot\Scripts directory and add the simple code to run the prompt using model:llama3.2

from langchain_ollama import OllamaLLM

model=OllamaLLM(model="llama3.2")

result=model.invoke(input="What is LLM")

print(result)

Here is the response from Visual Studio environment:









Popular posts from this blog

Connecting Claude to Pega Infinity 25.1.3 via MCP — Step-by-Step

-
Image
Enable AI-powered access to your Pega Investigative Case Management data directly from Claude prompts using the Model Context Protocol (MCP) and OAuth 2.0. What is MCP and why connect Pega to Claude? The Model Context Protocol (MCP) is an open standard that lets AI assistants like Claude securely call tools and read data from external systems — all from a natural-language prompt. Think of it as a universal adapter between your enterprise systems and AI. Pega Infinity V25.1.3 ships with a first-class MCP service built on its existing REST framework. Once enabled, Claude can: Query and summarize investigative cases without switching apps Retrieve assignment details and case status on demand Trigger Pega workflows from conversational prompts Version requirement: This guide is based on Pega Infinity V25.1.3 (Community Edition) with the Rule-Service-MCP rule type. Earlier versions may not include native MCP support. Architecture overview The connection uses three layers: Claude as the AI c...
Read more

Build Pega Applications with GitHub Copilot CLI, Pega Infinity AI Plugin & MCP — Branch-Based Agentic Authoring

-
Image
To work on this experiment, we need the Github Copilot CLI, Pega Infinity Instance, and Pega Infinity AI Plugins ( https://github.com/pegasystems/infinity-ai-plugins ) . In my case, I have used the Pega Infinity V26.1 Community Edition and here is the Github repository to upload it in the Github Copilot plugin marketplace. For this entire exercise to work well, we need Java 17 or above.  Windows Command Prompt (Agent-Driven Terminal): Represents the Github Copilot CLI. It shows the active terminal execution where commands like pega-setup, list-skills, list-available-applications etc. MCP (Model Context Protocol) Orchestrator: Acts as the bridge and mediator between the AI terminal environment (GitHub Copilot) and the Pega platform. It translates natural language directives and agent commands into structured operations and API calls to manage application configurations. Here the name of the MCP server is pega--infinity-authoring Workflow Directives: Represents...
Read more

itextpdf API to generate PDF doc from an image file using Pega PE

-
Image
Probably some of the techies might have an idea on an opensource called " itextpdf " (earlier name is lowagie ) to convert an image file(png,jpeg,jpg,tiff,gif etc.) to a PDF file. It is very simple to use this API in Pega function to build the logic using java code. Moreover, this API comes in one single jar file and we need to import it in Pega before we start using it.  I see the conversion comes good in PDF file but not sure if any limitations of this API in terms of conversion is like formatting, font, images etc. So far so good and may be we can keep this API as one of our solutions while working on the requirements accordingly.  Heard that same API can be used to convert the text file to PDF but I didn't explore on it. Those who are interested to work on text -> PDF can take a look at the documentation. Don't hesitate to contact me offline for any further discussion in this context :) :) Prerequisites for doing this exercise: 1) itextpdf 5.4.0 jar file 2) Peg...
Read more