NCRF++, a Neural Sequence Labeling Toolkit. Easy use to any sequence labeling tasks (e.g. NER, POS, Segmentation). It includes character LSTM/CNN, word LSTM/CNN and softmax/CRF components.
A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.
Fully neural approach for text chunking
The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.
A package for parsing PDFs and analyzing their content using LLMs.
A TensorFlow implementation of Neural Sequence Labeling model, which is able to tackle sequence labeling tasks such as POS Tagging, Chunking, NER, Punctuation Restoration and etc.
PDFStract - The Extraction and Chunking Layer in Your RAG Pipeline - Available as CLI - WEBUI - API
A Python CLI to test, benchmark, and find the best RAG chunking strategy for your Markdown documents.
An LLM GUI application; enables you to interact with your files, offering dynamic parameters that can modify response behavior during runtime.
smart-llm-loader is a lightweight yet powerful Python package that transforms any document into LLM-ready chunks. Spend less time on preprocessing headaches and more time building what matters. From RAG systems to chatbots to document Q&A, SmartLLMLoader handles the heavy lifting so you can focus on creating exceptional AI applications.
FastCDC implementation in Python https://pypi.org/project/fastcdc/
One library to split them all: Sentence, Code, Docs. Chunk smarter, not harder -- built for LLMs, RAG pipelines, and beyond.
Extract and align grammar patterns from English sentences.
LLM Chatbot w/ Retrieval Augmented Generation using Llamaindex. It demonstrates how to impl. chunking, indexing, and source citation.
Add a description, image, and links to the chunking topic page so that developers can more easily learn about it.
To associate your repository with the chunking topic, visit your repo's landing page and select "manage topics."