Free open source alternative app list about topic llm - TopFreeAlternative

Free open source alternative app list about topic "llm" :

Quickstart | Configurations | MacOS | Example notebooks | FAQ AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run Kimi K3 (2.8T) — the largest open-source model released to date — on under 4GB , Qwen3.8-Flash-Next (125B) on 6GB , and DeepSeek-V3 (671B) on ~12GB . We now also support training huge models on small VRAM: Qwen3.8-Flash-Next (125B) under 6GB . AI Agents Recommendation: Best AI Game Sprite Generator Best AI Facial Expression Editor Updates [2026/09] Training support: stream frozen weights one layer at a time and keep adapters on the GPU. Qwen3.8-Flash-Next (125B) trains under 6GB (RTX 3060 Ti); Qwen3.8-27B trains in ~2GB at seq 512. See Training . [2026/08] Qwen3.8-Flash-Next support: Qwen's 125B MoE flagship ( Qwen4ExpForConditionalGeneration ) with a ~51B n-gram embedding runs in 5.95GB of VRAM, measured end to end on one RTX 4090. The n-gram table is file-mapped on the host (a 64GB machine is enough); decoder layers stream. Needs a transformers build with in-tree qwen4_exp ( pip install git+https://github.com/huggingface/transformers.git today) and ~360GB of checkpoint disk ( delete_original=True reclaims the originals after the split). [2026/08] Qwen3.8-27B support: Qwen's new dense VL (Gated DeltaNet + Gated Attention, native vision) runs in 3.33GB of VRAM, measured end to end on one RTX 3090. Needs transformers 5.8+. [2026/07] Kimi K3 (2.8T) support: the largest open-source model runs on a single card in 3.72GB of VRAM, measured end to end on one RTX 6000 Ada. Per-expert streaming loads only the experts a token actually routes to. K3 brings three requirements of its own: pip install compressed-tensors flash-attn (its model code mandates flash attention regardless of what you request), a CUDA 12 build of torch, since no prebuilt flash-attn wheel exists for CUDA 13 yet, and transformers 4.56.x, as its remote code does not load on 5.x. [2026/06] v3.0 : FP8 model support + the latest models. Run DeepSeek-V3 (671B) on ~12GB and Qwen3-235B on ~3GB , plus Qwen3, Llama 3.x/4, DeepSeek V2/V3, Phi-4, Gemma and more — all through a single AutoModel . [2024/08/20] v2.11.0: Support Qwen2.5 [2024/08/18] v2.10.1 Support CPU inference. Support non sharded models. Thanks @NavodPeiris for the great work! [2024/07/30] Support Llama3.1 405B ( example notebook ). Support 8bit/4bit quantization . [2024/04/20] AirLLM supports Llama3 natively already. Run Llama3 70B on 4GB single GPU. [2023/12/25] v2.8.2: Support MacOS running 70B large language models. [2023/12/20] v2.7: Support AirLLMMixtral. [2023/12/20] v2.6: Added AutoModel, automatically detect model type, no need to provide model class to initialize model. [2023/12/18] v2.5: added prefetching to overlap the model loading and compute. 10% speed improvement. [2023/12/03] added support of ChatGLM , QWen , Baichuan , Mistral , InternLM ! [2023/12/02] added support for safetensors. Now support all top 10 models in open llm leaderboard. [2023/12/01] airllm 2.0. Support compressions: 3x run time speed up! [2023/11/20] airllm Initial version! Star History Table of Contents Quick start Model Compression Configurations Run on MacOS Example notebooks Supported Models Training Acknowledgement FAQ Quickstart 1. Install package First, install the airllm pip package. pip install airllm 2. Inference Then, initialize AirLLMLlama2, pass in the huggingface repo ID of the model being used, or the local path, and inference can be performed similar to a regular transformer model. ( You can also specify the path to save the splitted layered model through layer_shards_saving_path when init AirLLMLlama2. from airllm import AutoModel MAX_LENGTH = 128 # just pass a hugging face repo id — works with almost any popular model: model = AutoModel . from_pretrained ( "Qwen/Qwen3-32B" ) # go bigger with the exact same one line: #model = AutoModel.from_pretrained("Qwen/Qwen3.8-27B") # 27B dense VL, 3.33GB #model = AutoModel.from_pretrained("Qwen/Qwen3.8-Flash-Next") # 125B MoE + 51B PLE, 5.95GB #model = AutoModel.from_pretrained("Qwen/Qwen3-235B-A22B") # 235B, runs in ~3GB #model = AutoModel.from_pretrained("deepseek-ai/DeepSeek-V3") # 671B, runs in ~12GB # or use a model's local path... #model = AutoModel.from_pretrained("/home/ubuntu/.cache/huggingface/hub/models--Qwen--Qwen3-32B/snapshots/...") input_text = [ 'What is the capital of United States?' , #'I like', ] input_tokens = model . tokenizer ( input_text , return_tensors = "pt" , return_attention_mask = False , truncation = True , max_length = MAX_LENGTH , padding = False ) generation_output = model . generate ( input_tokens [ 'input_ids' ]. cuda (), max_new_tokens = 20 , use_cache = True , return_dict_in_generate = True ) output = model . tokenizer . decode ( generation_output . sequences [ 0 ]) print ( output ) Note: During inference, the original model will first be decomposed and saved layer-wise. Please ensure there is sufficient disk space in the huggingface cache directory. Model Compression - 3x Inference Speed Up! We just added model compression based on block-wise quantization-based model compression. Which can further speed up the inference speed for up to 3x , with almost ignorable accuracy loss! (see more performance evaluation and why we use block-wise quantization in this paper ) How to enable model compression speed up: Step 1. make sure you have bitsandbytes installed by pip install -U bitsandbytes Step 2. make sure airllm verion later than 2.0.0: pip install -U airllm Step 3. when initialize the model, passing the argument compression ('4bit' or '8bit'): model = AutoModel . from_pretrained ( "garage-bAInd/Platypus2-70B-instruct" , compression = '4bit' # specify '8bit' for 8-bit block-wise quantization ) What are the differences between model compression and quantization? Quantization normally needs to quantize both weights and activations to really speed things up. Which makes it harder to maintain accuracy and avoid the impact of outliers in all kinds of inputs. While in our case the bottleneck is mainly at the disk loading, we only need to make the model loading size smaller. So, we get to only quantize the weights' part, which is easier to ensure the accuracy. Configurations When initialize the model, we support the following configurations: compression : supported options: 4bit, 8bit for 4-bit or 8-bit block-wise quantization, or by default None for no compression profiling_mode : supported options: True to output time consumptions or by default False layer_shards_saving_path : optionally another path to save the splitted model hf_token : huggingface token can be provided here if downloading gated models like: meta-llama/Llama-2-7b-hf prefetching : prefetching to overlap the model loading and compute. By default, turned on. For now, only AirLLMLlama2 supports this. delete_original : if you don't have too much disk space, you can set delete_original to true to delete the original downloaded hugging face model, only keep the transformed one to save half of the disk space. MacOS Just install airllm and run the code the same as on linux. See more in Quick Start . make sure you installed mlx and torch you probably need to install python native see more here only Apple silicon is supported Example [python notebook] ( https://github.com/lyogavin/airllm/blob/main/air_llm/examples/run_on_macos.ipynb ) Example Python Notebook Example colabs here: example of other models (ChatGLM, QWen, Baichuan, Mistral, etc): Details ChatGLM: from airllm import AutoModel MAX_LENGTH = 128 model = AutoModel . from_pretrained ( "THUDM/chatglm3-6b-base" ) input_text = [ 'What is the capital of China?' ,] input_tokens = model . tokenizer ( input_text , return_tensors = "pt" , return_attention_mask = False , truncation = True , max_length = MAX_LENGTH , padding = True ) generation_output = model . generate ( input_tokens [ 'input_ids' ]. cuda (), max_new_tokens = 5 , use_cache = True , return_dict_in_generate = True ) model . tokenizer . decode ( generation_output . sequences [ 0 ]) QWen: from airllm import AutoModel MAX_LENGTH = 128 model = AutoModel . from_pretrained ( "Qwen/Qwen-7B" ) input_text = [ 'What is the capital of China?' ,] input_tokens = model . tokenizer ( input_text , return_tensors = "pt" , return_attention_mask = False , truncation = True , max_length = MAX_LENGTH ) generation_output = model . generate ( input_tokens [ 'input_ids' ]. cuda (), max_new_tokens = 5 , use_cache = True , return_dict_in_generate = True ) model . tokenizer . decode ( generation_output . sequences [ 0 ]) Baichuan, InternLM, Mistral, etc: from airllm import AutoModel MAX_LENGTH = 128 model = AutoModel . from_pretrained ( "baichuan-inc/Baichuan2-7B-Base" ) #model = AutoModel.from_pretrained("internlm/internlm-20b") #model = AutoModel.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") input_text = [ 'What is the capital of China?' ,] input_tokens = model . tokenizer ( input_text , return_tensors = "pt" , return_attention_mask = False , truncation = True , max_length = MAX_LENGTH ) generation_output = model . generate ( input_tokens [ 'input_ids' ]. cuda (), max_new_tokens = 5 , use_cache = True , return_dict_in_generate = True ) model . tokenizer . decode ( generation_output . sequences [ 0 ]) To request other model support: here Supported Models AirLLM works out of the box with virtually every popular open LLM — just pass its Hugging Face ID to AutoModel.from_pretrained(...) . That covers all the major families: Llama (2 / 3 / 3.1 / 3.3 / 4) · Qwen (1 / 2 / 2.5 / 3 / 3.5 / 3.8, including MoE, Flash-Next, FP8, and native VL) · DeepSeek (V2 / V3 / R1) · Mistral & Mixtral · Phi · Gemma · ChatGLM · Baichuan · InternLM · Yi · Kimi K3 — and most new models the day they're released. Tiny GPU, huge models The trick: AirLLM only ever keeps one layer on the GPU at a time , so the VRAM you need depends on the model's layer size — not its total size. That's how a 671B model fits on a hobbyist card: Model Size GPU VRAM Qwen3 / Mistral / Phi (≈8B) 8B ~1–2 GB Qwen3-30B / Mixtral (MoE) 30–47B ~1–3 GB Qwen3.8-27B (dense VL) 27B 3.33 GB Qwen3.8-Flash-Next (MoE + PLE) ~180B 5.95 GB Qwen3-235B (MoE) 235B ~3 GB Llama 3.x 70B (full precision) 70B ~4 GB Llama 3.1 405B 405B ~8 GB DeepSeek-V3 671B ~12 GB Same one line of code for all of them — no special setup. Training AirLLM can fine-tune huge models on a small GPU. Frozen base weights stream from disk one decoder layer at a time; only the adapters stay resident. Qwen3.8-Flash-Next (125B) trains under 6GB ; Qwen3.8-27B trains in ~2GB at seq 512. This is not Hugging Face Trainer / bitsandbytes QLoRA. Flash-Next needs a transformers build with in-tree qwen4_exp ( pip install git+https://github.com/huggingface/transformers.git today). 1. Prepare a dataset One JSON object per line ( .jsonl ). The usual field is text — next-token prediction over the whole string: { "text" : " Your first training document. Can be a few sentences or a few paragraphs. " } { "text" : " Your second training document. " } Instruction pairs work too. Loss is applied on the completion only: { "prompt" : " What is AirLLM? " , "completion" : " A library that runs and trains huge models on small VRAM. " } { "instruction" : " Translate to English " , "input" : " bonjour " , "output" : " hello " } A .txt file is also fine: one example per blank-line-separated block. A two-line starter file lives at air_llm/examples/sft_example.jsonl . 2. Run training From the repo root, point --data at your file: python air_llm/examples/train_qwen38_flash_next_lora.py \ --data my_data.jsonl \ --seq-len 512 \ --epochs 1 \ --save-adapter qwen38-flash-next-lora.pt For the 27B dense model: python air_llm/examples/train_qwen38_lora.py \ --data my_data.jsonl \ --seq-len 512 \ --epochs 1 \ --save-adapter qwen38-27b-lora.pt --steps N stops after N examples (useful for a smoke test). Omit --data and the script overfits a built-in snippet. Python API from airllm import AirLLMLoRAQwen4Exp trainer = AirLLMLoRAQwen4Exp ( "Qwen/Qwen3.8-Flash-Next" , max_seq_len = 512 , lora_r = 16 , delete_original = True , ) tok = trainer . tokenizer if tok . pad_token_id is None : tok . pad_token = tok . eos_token encoded = tok ( "Your training text here." , return_tensors = "pt" , truncation = True , max_length = 512 , ) loss = trainer . train_step ( encoded [ "input_ids" ]. cuda (), attention_mask = encoded . get ( "attention_mask" ), ) print ( loss ) trainer . save_adapter ( "qwen38-flash-next-lora.pt" ) AirLLMLoRA is the same API for Qwen/Qwen3.8-27B . Acknowledgement A lot of the code are based on SimJeg's great work in the Kaggle exam competition. Big shoutout to SimJeg: GitHub account @SimJeg , the code on Kaggle , the associated discussion . FAQ 1. MetadataIncompleteBuffer safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer If you run into this error, most possible cause is you run out of disk space. The process of splitting model is very disk-consuming. See this . You may need to extend your disk space, clear huggingface .cache and rerun. 2. ValueError: max() arg is an empty sequence Most likely you are loading QWen or ChatGLM model with Llama2 class. Try the following: For QWen model: from airllm import AutoModel #<----- instead of AirLLMLlama2 AutoModel . from_pretrained (...) For ChatGLM model: from airllm import AutoModel #<----- instead of AirLLMLlama2 AutoModel . from_pretrained (...) 3. 401 Client Error....Repo model ... is gated. Some models are gated models, needs huggingface api token. You can provide hf_token: model = AutoModel . from_pretrained ( "meta-llama/Llama-2-7b-hf" , #hf_token='HF_API_TOKEN') 4. ValueError: Asking to pad but the tokenizer does not have a padding token. Some model's tokenizer doesn't have padding token, so you can set a padding token or simply turn the padding config off: input_tokens = model . tokenizer ( input_text , return_tensors = "pt" , return_attention_mask = False , truncation = True , max_length = MAX_LENGTH , padding = False #<----------- turn off padding ) Citing AirLLM If you find AirLLM useful in your research and wish to cite it, please use the following BibTex entry: @software{airllm2023, author = {Gavin Li}, title = {AirLLM: scaling large language models on low-end commodity computers}, url = {https://github.com/lyogavin/airllm/}, version = {0.0}, year = {2023}, } Contribution Welcomed contributions, ideas and discussions! If you find it useful, please ⭐ or buy me a coffee! 🙏

FreeWindowsmacOSLinux

🚀🤖 Crawl4AI: the open-source web crawler for LLMs and AI agents Latest: v0.9.4 (23 Sep 2026) · all releases → Crawl4AI turns any website into clean, LLM-ready Markdown for RAG, AI agents and data pipelines. Run the open-source web crawler and scraper yourself, free forever, or use it hosted with one key: scrape, search and extract through one API, with MCP for your agent. Two ways to use Crawl4AI 🐍 Run it yourself: open source, forever pip install -U crawl4ai crawl4ai-setup # installs the browser, once import asyncio from crawl4ai import AsyncWebCrawler async def main (): async with AsyncWebCrawler () as crawler : result = await crawler . arun ( url = "https://news.ycombinator.com" ) print ( result . markdown ) asyncio . run ( main ()) Docker server, CLI and every option: Installation · docs.crawl4ai.com ☁️ Or use the cloud: no browsers, no proxies Verify your email and free credit to start is yours. No card. Soft launch: prices can change, what you buy stays yours. Get any page as Markdown: curl -s https://api.crawl4ai.com/scrape \ -H " Authorization: Bearer $CRAWL4AI_KEY " \ -H " Content-Type: application/json " \ -d ' {"url": "https://news.ycombinator.com"} ' | jq -r .markdown The same key works for /search , /answer , /extract and many URLs at once ( /scrape/batch , /scrape/jobs ). Pay as you go: live prices . Give it to your AI agent. Claude Code shown; Codex, Cursor and OpenCode → claude mcp add --transport http crawl4ai https://api.crawl4ai.com/mcp \ --header " Authorization: Bearer $CRAWL4AI_KEY " Which one? 🐍 Library 🐳 Your own server ☁️ Crawl4AI Cloud Runs the browsers you, in your Python process you, in Docker on your machine we do JS-heavy pages and bot walls your settings, your proxies your settings, your proxies handled for you, automatically Web search – – /search and /answer Price free, forever free (your hosting) pay as you go; free credit to start 🤓 My Personal Story I grew up on an Amstrad, thanks to my dad, and never stopped building. In grad school I specialized in NLP and built crawlers for research. That’s where I learned how much extraction matters. In 2023, I needed web-to-Markdown. The “open source” option wanted an account, API token, and $16, and still under-delivered. I went turbo anger mode, built Crawl4AI in days, and it went viral. Now it’s the most-starred crawler on GitHub. I made it open source for availability , anyone can use it without a gate. Now I’m building the platform for affordability , anyone can run serious crawls without breaking the bank. If that resonates, join in, send feedback, or just crawl something amazing. That platform is live now: Crawl4AI Cloud . Why developers pick Crawl4AI LLM-ready output : smart Markdown with headings, tables, code and citation hints Fast in practice : async browser pool, caching, minimal hops Full control : sessions, proxies, cookies, user scripts, hooks Adaptive intelligence : learns site patterns, explores only what matters Deploy anywhere : no keys needed, CLI and Docker, or the hosted cloud ✨ Features 📝 Markdown generation 🧹 Clean Markdown : headings, lists, tables and code blocks, in a structure an LLM reads well. 🎯 Fit Markdown : filters remove menus, footers and boilerplate: PruningContentFilterLXML , BM25ContentFilter (for a query) and LLMContentFilter . 🔗 Citations : page links become a numbered reference list. 🛠️ Your own strategy : plug in a custom Markdown generator. ☁️ Same in the cloud: POST /scrape returns this Markdown, with no browser to run. Docs → 📊 Structured data extraction 🔎 CSS and XPath schemas : fast extraction with no LLM ( JsonCssExtractionStrategy , JsonXPathExtractionStrategy , RegexExtractionStrategy ). 🪄 Schema generator : describe what you want once; generate_schema writes a reusable schema. 🤖 LLM extraction : any LLM provider, open-source or hosted, into a typed JSON schema ( LLMExtractionStrategy ). 🧱 Chunking : topic, regex and sentence chunking for long pages. 🌌 Cosine similarity : find the chunks that match a query ( CosineStrategy ). ☁️ Same in the cloud: POST /extract , with no LLM key of your own. Docs → 🌐 Browser control 🖥️ Your own browser : persistent profiles with saved logins, cookies and settings. 🔄 Remote browsers : connect over the Chrome DevTools Protocol (CDP). 🔒 Sessions : keep a browser state across multi-step crawls. 🧩 Proxies : with authentication and rotation. 🕶️ Stealth mode : enable_stealth , and an undetected-browser adapter for sites that detect automation. ⚙️ Full control : headers, cookies, user agents, viewport. 🌍 Chromium, Firefox and WebKit . 🔎 Crawling and scraping 🕸️ Deep crawl : BFS, DFS and best-first strategies, with crash recovery ( resume_state ) for long crawls. 🧠 Adaptive crawling : AdaptiveCrawler stops when it has learned enough to answer your query. 🌱 URL discovery : AsyncUrlSeeder (sitemaps, Common Crawl) and DomainMapper ; prefetch=True finds URLs 5 to 10 times faster. 🚀 Dynamic pages : run JavaScript, wait for elements, scroll the full page ( scan_full_page ) for infinite scroll and lazy images. 📸 Screenshots and PDFs of any page. 🖼️ Media and links : images, audio, video, srcset , internal and external links, iframes, metadata. 📂 Raw HTML and local files : raw: and file:// . 🛠️ Hooks at every step of a crawl. 💾 Caching to skip repeated fetches. ⚡ Many URLs at once : arun_many with a memory-adaptive dispatcher. ☁️ Same in the cloud: up to 50 URLs in one streamed call, or 10,000 in a background job. Docs → 🐳 Self-hosting (Docker) 🔐 Secure by default : every endpoint needs your CRAWL4AI_API_TOKEN . 🧰 REST API : /md , /html , /crawl , /crawl/stream , /screenshot , /pdf , /execute_js . 🤖 MCP : connect Claude Code and other agents to your own server. 📊 Monitoring dashboard and playground , a browser pool with pre-warmed pages. 🏗️ AMD64 and ARM64 images. ☁️ Rather not run a server? The cloud is the same idea, hosted. Get a key → ☁️ What the cloud adds 🔍 Web search API : GET /search , browser-free, ranked and cleaned. Docs → 💬 Answers : GET /answer gives a direct answer to a question (experimental). Docs → 🧪 Extraction without your own LLM key : POST /extract . Docs → 🧗 JS-heavy pages and bot walls : handled automatically; you never pick an engine. Docs → 🤝 MCP for your agent : one line in Claude Code, Codex, Cursor or OpenCode. Docs → 🛠️ Installation 🐍 pip pip install -U crawl4ai crawl4ai-setup # installs and sets up the browser crawl4ai-doctor # checks the installation If the browser setup fails, install it by hand: python -m playwright install --with-deps chromium Pre-release versions: pip install crawl4ai --pre Development install , for contributors: git clone https://github.com/unclecode/crawl4ai.git cd crawl4ai pip install -e " .[all] " # or: pip install -e . (the core only) 🐳 Docker server The server needs a token. Without one it answers only inside its container. export CRAWL4AI_API_TOKEN= " $( openssl rand -hex 32 ) " docker run -d -p 11235:11235 --name crawl4ai --shm-size=1g \ -e CRAWL4AI_API_TOKEN= " $CRAWL4AI_API_TOKEN " \ unclecode/crawl4ai:latest Test it (allow about 10 seconds for the start): curl -s http://localhost:11235/md \ -H " Authorization: Bearer $CRAWL4AI_API_TOKEN " \ -H " Content-Type: application/json " \ -d ' {"url": "https://news.ycombinator.com"} ' | jq -r .markdown The dashboard is at http://localhost:11235/dashboard , the playground at http://localhost:11235/playground . LLM keys, MCP and every setting: Self-hosting guide . ⌨️ Command line (`crwl`) # A page as Markdown crwl https://news.ycombinator.com -o markdown # Deep crawl, breadth first, at most 10 pages crwl https://docs.crawl4ai.com --deep-crawl bfs --max-pages 10 # Ask a question about a page (needs an LLM key: crwl config) crwl https://www.example.com/products -q " Extract all product prices " 🔬 Advanced usage examples More in docs/examples . 📝 Clean and fit Markdown import asyncio from crawl4ai import AsyncWebCrawler , BrowserConfig , CrawlerRunConfig , CacheMode from crawl4ai . content_filter_strategy import PruningContentFilterLXML from crawl4ai . markdown_generation_strategy import DefaultMarkdownGenerator async def main (): run_config = CrawlerRunConfig ( cache_mode = CacheMode . BYPASS , markdown_generator = DefaultMarkdownGenerator ( content_filter = PruningContentFilterLXML ( threshold = 0.48 , threshold_type = "fixed" , min_word_threshold = 0 ) ), ) async with AsyncWebCrawler ( config = BrowserConfig ( headless = True )) as crawler : result = await crawler . arun ( url = "https://en.wikipedia.org/wiki/Web_crawler" , config = run_config ) print ( len ( result . markdown . raw_markdown ), "characters of raw Markdown" ) print ( len ( result . markdown . fit_markdown ), "characters after the filter" ) asyncio . run ( main ()) 🖥️ A JavaScript page and structured data, without an LLM import asyncio , json from crawl4ai import AsyncWebCrawler , BrowserConfig , CrawlerRunConfig , CacheMode , JsonCssExtractionStrategy schema = { "name" : "Quotes" , "baseSelector" : "div.quote" , "fields" : [ { "name" : "text" , "selector" : "span.text" , "type" : "text" }, { "name" : "author" , "selector" : "small.author" , "type" : "text" }, { "name" : "tags" , "selector" : "a.tag" , "type" : "list" , "fields" : [{ "name" : "tag" , "type" : "text" }]}, ], } async def main (): run_config = CrawlerRunConfig ( extraction_strategy = JsonCssExtractionStrategy ( schema ), scan_full_page = True , # scroll to the end, so the page loads every quote scroll_delay = 0.5 , cache_mode = CacheMode . BYPASS , ) async with AsyncWebCrawler ( config = BrowserConfig ( headless = True )) as crawler : result = await crawler . arun ( url = "https://quotes.toscrape.com/scroll" , config = run_config ) quotes = json . loads ( result . extracted_content ) print ( f"Extracted { len ( quotes ) } quotes" ) print ( json . dumps ( quotes [ 0 ], indent = 2 )) asyncio . run ( main ()) 📚 Structured data with an LLM import os , asyncio from pydantic import BaseModel , Field from crawl4ai import AsyncWebCrawler , CrawlerRunConfig , CacheMode , LLMConfig , LLMExtractionStrategy class ModelFee ( BaseModel ): model_name : str = Field (..., description = "Name of the model." ) input_fee : str = Field (..., description = "Fee for input tokens." ) output_fee : str = Field (..., description = "Fee for output tokens." ) async def main (): run_config = CrawlerRunConfig ( cache_mode = CacheMode . BYPASS , extraction_strategy = LLMExtractionStrategy ( # any provider LiteLLM supports, e.g. "ollama/llama3.3" with api_token="no-token" llm_config = LLMConfig ( provider = "openai/gpt-4o-mini" , api_token = os . getenv ( "OPENAI_API_KEY" )), schema = ModelFee . model_json_schema (), extraction_type = "schema" , instruction = "Extract every model name with its input and output token fee." , ), ) async with AsyncWebCrawler () as crawler : result = await crawler . arun ( url = "https://openai.com/api/pricing/" , config = run_config ) print ( result . extracted_content ) asyncio . run ( main ()) 🤖 Your own browser with a saved profile import os , asyncio from pathlib import Path from crawl4ai import AsyncWebCrawler , BrowserConfig , CrawlerRunConfig , CacheMode async def main (): user_data_dir = os . path . join ( Path . home (), ".crawl4ai" , "browser_profile" ) os . makedirs ( user_data_dir , exist_ok = True ) browser_config = BrowserConfig ( headless = True , user_data_dir = user_data_dir , use_persistent_context = True ) run_config = CrawlerRunConfig ( cache_mode = CacheMode . BYPASS , magic = True ) async with AsyncWebCrawler ( config = browser_config ) as crawler : result = await crawler . arun ( url = "ADDRESS_OF_A_CHALLENGING_WEBSITE" , config = run_config ) print ( result . success , len ( result . markdown )) asyncio . run ( main ()) 📖 Documentation Library docs, guides and API reference: docs.crawl4ai.com Cloud docs: crawl4ai.com/docs Release notes: releases · Roadmap: ROADMAP.md 🤝 Contributing We welcome contributions from the open-source community. Check out our contribution guidelines for more information. 📄 License & Attribution This project is licensed under the Apache License 2.0, attribution is recommended via the badges below. See the Apache 2.0 License file for details. Attribution Requirements When using Crawl4AI, you must include one of the following attribution methods: 📈 1. Badge Attribution (Recommended) Add one of these badges to your README, documentation, or website: Theme Badge Disco Theme (Animated) Night Theme (Dark with Neon) Dark Theme (Classic) Light Theme (Classic) HTML code for adding the badges:  < a href =" https://github.com/unclecode/crawl4ai " > < img src =" https://raw.githubusercontent.com/unclecode/crawl4ai/main/docs/assets/powered-by-disco.svg " alt =" Powered by Crawl4AI " width =" 200 " /> </ a >  < a href =" https://github.com/unclecode/crawl4ai " > < img src =" https://raw.githubusercontent.com/unclecode/crawl4ai/main/docs/assets/powered-by-night.svg " alt =" Powered by Crawl4AI " width =" 200 " /> </ a >  < a href =" https://github.com/unclecode/crawl4ai " > < img src =" https://raw.githubusercontent.com/unclecode/crawl4ai/main/docs/assets/powered-by-dark.svg " alt =" Powered by Crawl4AI " width =" 200 " /> </ a >  < a href =" https://github.com/unclecode/crawl4ai " > < img src =" https://raw.githubusercontent.com/unclecode/crawl4ai/main/docs/assets/powered-by-light.svg " alt =" Powered by Crawl4AI " width =" 200 " /> </ a >  < a href =" https://github.com/unclecode/crawl4ai " > < img src =" https://img.shields.io/badge/Powered%20by-Crawl4AI-blue?style=flat-square " alt =" Powered by Crawl4AI " /> </ a > 📖 2. Text Attribution Add this line to your documentation: ``` This project uses Crawl4AI ( https://github.com/unclecode/crawl4ai ) for web data extraction. ``` 📚 Citation If you use Crawl4AI in your research or project, please cite: @software { crawl4ai2024 , author = { UncleCode } , title = { Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper } , year = { 2024 } , publisher = { GitHub } , journal = { GitHub Repository } , howpublished = { \url{https://github.com/unclecode/crawl4ai} } , commit = { Please use the commit hash you're working with } } Text citation format: UncleCode. (2024). Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper [Computer software]. GitHub. https://github.com/unclecode/crawl4ai 🗾 Mission Our mission is to unlock the value of personal and enterprise data by turning digital footprints into structured, useful assets. Crawl4AI gives individuals and organizations open-source tools to extract and structure data, and a fair way to benefit from it. Full mission statement → 💖 Support Crawl4AI ⭐ Star the repo : it helps more people find it. ☁️ Use the cloud : crawl4ai.com . It funds the library. 💝 Sponsor on GitHub : github.com/sponsors/unclecode 🏢 Companies : the sponsor tiers and benefits are in SPONSORS.md . 🌟 Current Sponsors 🤝 Strategic Partners These companies provide core infrastructure and technology that power Crawl4AI’s capabilities — from web access and proxy networks to AI tooling and data pipelines. Company About Massive is a web access API backed by millions of volunteer devices in 195+ countries. AI agents, models, and data pipelines use it to reach any site on the internet, reliably, in real time, and at scale. 🏢 Enterprise Sponsors Our enterprise sponsors support Crawl4AI and help scale it to power production-grade data pipelines. Company About Sponsorship Tier Helps engineers and buyers find, compare, and source electronic & industrial parts in seconds, with specs, pricing, lead times & alternatives. 🥇 Gold Kidocode is a hybrid technology and entrepreneurship school for kids aged 5–18, offering both online and on-campus education. 🥇 Gold Singapore-based Aleph Null is Asia’s leading edtech hub, dedicated to student-centric, AI-driven education—empowering learners with the tools to thrive in a fast-changing world. 🥇 Gold 💼 Become a Strategic Partner or Sponsor Interested in partnering with Crawl4AI? Whether you’re a proxy provider, AI infrastructure company, cloud platform, or an organization looking to support the Crawl4AI ecosystem, we’d love to hear from you. 📩 Contact: [email protected] 🧑‍🤝 Individual Sponsors A heartfelt thanks to our individual supporters! Every contribution helps us keep our opensource mission alive and thriving! Want to join them? Sponsor Crawl4AI → 📧 Contact Discord · X @unclecode · GitHub @unclecode · [email protected] Building crawlers or AI agents for a living? DM me on X. I want to work with people like you, and we are hiring. From a company? We have an enterprise offer and we tailor it to your business. SOC 2 Type I is done, Type II is in progress. Write to [email protected] . Happy crawling! 🕸️🚀 Star History

FreeWindowsmacOSLinux

Key features Self-hosted AI chat interface that you can run on your own hardware, keeping conversations and data under your control. Works with both local models (for example via Ollama) and remote LLM providers from a single web UI. Extensible, community-driven project designed to be customized and adapted to different workflows. Multi-model support, letting you switch between different language models without leaving the app. Browser-based chat experience that requires no client installation beyond a modern web browser. Platforms: Windows, macOS, Linux. Links Official website Source code Pricing: Fully free and open source under the MIT license; self-hostable Docker image with no paid tier.

FreeWindowsmacOSLinux

Key features Native macOS and iOS application built specifically for Apple platforms. Chat interface designed for interacting with large language models. Runs conversations locally on-device, keeping data private. Focused on privacy by avoiding cloud-based processing. Tailored experience on both Mac and iPhone/iPad. Platforms: macOS, iOS Links Official website Pricing: Free to download; closed-source proprietary app.

FreemacOSiOS

Key features Open-source platform for building AI-powered knowledge bases and question-answering systems Uses Retrieval-Augmented Generation (RAG) to combine large language models with custom data sources Supports knowledge ingestion, indexing, and natural-language querying Vector search capabilities for retrieving relevant context from documents Chatbot workflow tools for creating conversational AI agents Flexible integration with various LLM providers Platforms: Linux, macOS, Windows Links Official Website Source Code Pricing: Free — self-hosted, free and open-source under the MIT license.

FreeWindowsmacOSLinux

Key features Enables AI agents to autonomously control and automate any web browser Supports LLM-driven browser tasks including navigation, form filling, and web scraping Provides a scalable framework for orchestrating complex browser interactions Built on Playwright for reliable cross-browser automation Distributed as a Python library for easy integration into AI workflows Open-source under the MIT license, allowing flexible self-hosting and customization Platforms: Windows, macOS, Linux Links Official Website Source Code Pricing Free and open-source under the MIT license; self-hosted Python library for AI browser automation.

FreeWindowsmacOSLinux

Key features Chat with your own documents using retrieval-augmented generation (RAG) and any supported LLM Desktop application with a single, unified interface for chatting, embedding, and managing vector storage Full support for local models, enabling fully offline and private use Works with a wide range of LLM providers, letting you swap models without changing your workflow Open source under the MIT license, giving you transparency and control over your data Platforms: Windows, macOS, Linux Links Official website Source code Pricing Fully free and open source under the MIT license, with optional paid cloud hosting tiers.

FreeWindowsmacOSLinux

Key features Open-source observability tool designed specifically for LLM applications. Provides tracing, metrics, and performance monitoring to help debug and optimize AI workflows. Self-hosted LLM observability with full telemetry capabilities. Licensed under Apache-2.0, enabling free use and modification. Platforms: Windows, macOS, Linux Links Official website: https://www.traceloop.com/openllmetry Source code: https://github.com/openllmetry/openllmetry Pricing: Free. OpenLLMetry is free and open-source under the Apache-2.0 license, offering self-hosted LLM observability with tracing and monitoring capabilities.

FreeWindowsmacOSLinux

Key features Open-source observability platform purpose-built for LLM applications. Combines monitoring, debugging, and analytics in one integrated tool. Native OpenTelemetry support for standardized telemetry collection. Self-hosted deployment with no usage limits. Licensed under Apache-2.0, so it's free to use and extend. A free, open-source alternative to commercial tools like LangSmith. Platforms Linux, macOS, Windows. Links Official website: https://openlit.ai Source code: https://github.com/openlit/openlit Pricing Free and open-source under Apache-2.0, self-hosted with no usage limits.

FreeWindowsmacOSLinux

Key features Free and open-source AI-powered research notebook. Privacy-first: runs locally on your own machine. Chat with PDFs, documents, and notes to gather insights. Bring your preferred LLM provider: Ollama, OpenAI-compatible APIs, or other providers. Self-host locally for full control over your data. Platforms: Windows, macOS, Linux Links Official website: https://open-notebook.vercel.app Source code: https://github.com/SiddharthMaheshwari/Open-Notebook Pricing: Free — completely free and open-source; self-host locally and connect to Ollama, OpenAI-compatible APIs, or other providers.

FreeWindowsmacOSLinux

Key features Run open-source large language models locally on your own machine. Provides a simple API and CLI for interacting with models. Offers a private and free alternative to cloud-based LLM services. Supported on Windows, macOS, and Linux. Platforms: Windows, macOS, Linux Links Official website: https://ollama.com Source code: https://github.com/ollama/ollama Pricing Free. Fully free and open-source under the MIT license; download and run any supported open-source LLM locally without restrictions.

FreeWindowsmacOSLinux

Key features Self-hosted AI runtime covering text, voice, vision, and agent capabilities. OpenAI-compatible API, making it a practical free alternative to the OpenAI API. Runs LLMs and other AI models locally, keeping data on your own hardware and off external services. Works across Windows, macOS, Linux, and Raspberry Pi OS. Fully open source under the MIT license — no usage limits or API fees. Platforms: Windows, macOS, Linux, Raspberry Pi OS Links Official website: https://localai.io Source code: https://github.com/mudler/LocalAI Pricing Free. Completely free and open source under the MIT license; self-host all AI features without usage limits or API fees.

FreeWindowsmacOSLinuxRaspberry Pi OS

llama.cpp is a free, open-source alternative to OpenAI . Key features Lightweight C++ implementation for running large language models locally. Supports a wide range of hardware backends. Quantized inference for efficient performance. Platforms: Windows, macOS, Linux Links Official website Source code Pricing: Free — fully open-source under the MIT license, free to use, modify, and distribute.

FreeWindowsmacOSLinux

Key features End-to-end tracing for LLM applications and AI agents in production. Monitoring and analytics tools built specifically for LLM observability. Self-hosted, open-source platform with full control over your data. Designed for AI-agent workloads, covering request-level visibility and debugging. Platforms: Linux, macOS, Windows. Links Official Website Source Code Pricing: Free. Open-source self-hosted observability for AI agents, including full tracing and analytics.

FreeWindowsmacOSLinux

Key features Simulation-based testing for AI agents and LLM applications. Evaluation tools for LLM applications. Observability for AI agents and LLM applications. Open-source platform licensed under Apache-2.0. Self-hostable via Docker. Platforms: Linux, macOS, Windows Links Official website: langwatch.ai Source code: github.com/langwatch/langwatch Pricing: Free and open-source under Apache-2.0, with self-hosting via Docker.

FreeWindowsmacOSLinux

Key features Open-source LLM engineering platform for AI-powered applications. Tracing and observability tools to debug and analyze LLM requests. Monitoring capabilities for production LLM-based products. Built-in evaluation features to assess and improve model performance. Prompt management to organize, version, and maintain prompts. Platforms: Linux, macOS, Windows Links Official website Source code repository Pricing Free. The entire platform is open source under the MIT license and free to self-host with no feature restrictions.

FreeWindowsmacOSLinux

Key features Open-source, AI-powered platform designed for engineering LLM products. Comprehensive LLM tracing to monitor and debug application behavior. Evaluation tools for assessing model performance and reliability. Prompt management features to organize and version prompts. Free to self-host with the full feature set, including tracing, evaluation, and prompt management. Available on Linux, macOS, and Windows. Platforms: Linux, macOS, Windows. Links Official website: https://www.lmnr.ai Source code: https://github.com/lmnr-ai/laminar Pricing: Free — the open-source platform is free to self-host with full features, including LLM tracing, evaluation, and prompt management.

FreeWindowsmacOSLinux

Key features Open-source LLM observability and monitoring platform built for AI applications. Tracks LLM requests to help developers understand usage patterns and behavior. Monitors costs associated with AI application usage. Measures latency and performance for model calls. Evaluates quality metrics to assess model outputs and application reliability. Platforms: Linux Links: Official website · Source code Pricing: Free self-hosted open-source license (Apache-2.0) with full observability features; cloud managed service available with a free tier.

FreeLinux

Key features Run local large language models directly on your own device, enabling fully private and offline AI chat without relying on cloud services. Keep all conversations and data on your hardware — nothing is uploaded, protecting your privacy. Free and open-source desktop application released under the permissive MIT license. No usage limits; all models execute locally on your own hardware. Cross-platform desktop support for Windows, macOS, and Linux. Platforms: Windows, macOS, Linux Links Official website: https://gpt4all.io Source code repository: https://github.com/nomic-ai/gpt4all Pricing Free. GPT4All is completely free and open-source (MIT) with no usage limits; all models run locally on your hardware. It serves as a private, offline alternative to cloud-based AI chat tools such as ChatGPT.

FreeWindowsmacOSLinux

prompts.chat The world's largest open-source prompt library for AI Works with ChatGPT, Claude, Gemini, Llama, Mistral, and more formerly known as Awesome ChatGPT Prompts 🌐 Browse Prompts • 📖 Read the Book • 📄 View on GitHub • 🚀 Self-Host 🏆 Featured in Forbes · 🎓 Referenced by Harvard , Columbia · 📄 40+ academic citations · ❤️ Most liked dataset on Hugging Face ⭐ 143k+ GitHub stars · 🏅 GitHub Staff Pick · 🚀 First prompt library (Dec 2022) Loved by AI pioneers: Greg Brockman (OpenAI Co-Founder) · Wojciech Zaremba (OpenAI Co-Founder) · Clement Delangue (Hugging Face CEO) · Thomas Dohmke (Former GitHub CEO) What is this? A curated collection of prompts for AI chat models. Originally created for ChatGPT, these prompts work great with any modern AI assistant. Browse Prompts Data Formats prompts.chat prompts.csv PROMPTS.md Hugging Face Dataset Want to contribute? Add prompts at prompts.chat/prompts/new — they sync here automatically. 📖 The Interactive Book of Prompting Learn prompt engineering with our free, interactive guide — 25+ chapters covering everything from basics to advanced techniques like chain-of-thought reasoning, few-shot learning, and AI agents. Start Reading → (Source: https://github.com/f/prompts.chat/tree/main/src/content/book ) 🎮 Prompting for Kids An interactive, game-based adventure to teach children (ages 8-14) how to communicate with AI through fun puzzles and stories. Start Playing → 🚀 Self-Hosting Deploy your own private prompt library with custom branding, themes, and authentication. Quick Start: npx prompts.chat new my-prompt-library cd my-prompt-library Manual Setup: git clone https://github.com/f/prompts.chat.git cd prompts.chat npm install && npm run setup The setup wizard configures branding, theme, authentication (GitHub/Google/Azure AD), and features. Recommended database: prompts.chat uses PostgreSQL. For a hosted database, we recommend Neon . Sponsored by 📖 Full Self-Hosting Guide • 🐳 Docker Guide 🔌 Integrations CLI npx prompts.chat Claude Code Plugin /plugin marketplace add f/prompts.chat /plugin install [email protected] 📖 Plugin Documentation MCP Server Use prompts.chat as an MCP server in your AI tools. Remote (recommended): { "mcpServers" : { "prompts.chat" : { "url" : " https://prompts.chat/api/mcp " } } } Local: { "mcpServers" : { "prompts.chat" : { "command" : " npx " , "args" : [ " -y " , " prompts.chat " , " mcp " ] } } } 📖 MCP Documentation 💖 Sponsors Built with Windsurf and Devin Become a Sponsor → 👥 Contributors 📜 License This project is dual-licensed: Source code and site-authored content is licensed under the MIT License , including the interactive book content in src/content/book . Prompt content and data (prompts.csv, PROMPTS.md, user-submitted prompts) is dedicated to the public domain under CC0 1.0 Universal . See LICENSE for details.

FreeWindowsmacOSLinux

Editor's Picks for Free Open-Source Software:

There's nothing here.

Featured on Twelve Tools

Featured on Startup Fame

Open-Launch Top 3 Daily Winner

View on SoloPush

Startup Fast Top 1 Daily Winner