abdulshakur/YT-TranscriptSegmenter

Transcript Segmenter

This tool intelligently segments long transcripts into smaller, manageable chunks that can be processed by LLM-based analyzers.

Features

• —Split transcripts at sentence boundaries to maintain coherence
• —Customize maximum segment length
• —Get detailed statistics about each segment
• —API endpoint for programmatic access
• —Works with any transcript (YouTube, meeting, podcast, etc.)

How to Use

Web Interface

1. 1.Paste your full transcript in the input box
2. 2.Adjust the maximum segment length if needed
3. 3.Keep "Use smart sentence boundaries" checked for better results
4. 4.Click "Segment Transcript"
5. 5.View and copy the segmented transcript

API Usage

This tool can be called programmatically using the Gradio client:

from gradio_client import Client

client = Client("https://your-space-name.hf.space")
segments, count, stats = client.predict(
    "Your full transcript text here",  # Input transcript
    1500,                              # Max segment length
    True,                              # Use smart boundaries
    fn_index=0
)

How It Works

The segmenter uses NLTK's sentence tokenizer to identify natural sentence boundaries in the text. It then builds segments that:

1. 1.Stay under the maximum token limit
2. 2.Preserve complete sentences whenever possible
3. 3.Include metadata about segment position

This approach creates more coherent segments that can be analyzed individually while maintaining context.

How It Works

The segmenter uses NLTK's sentence tokenizer to identify natural sentence boundaries in the text. It then builds segments that:

1. 1.Stay under the maximum token limit
2. 2.Preserve complete sentences whenever possible
3. 3.Include metadata about segment position

This approach creates more coherent segments that can be analyzed individually while maintaining context.

Integration with Analyzers

This segmenter is designed to work with LLM-based transcript analyzers. After segmenting a transcript, you can:

1. 1.Process each segment individually with an analyzer
2. 2.Combine the analyses for a complete understanding of the full transcript
3. 3.Avoid token limit errors that occur with long transcripts

Limitations

• —Works best when transcripts have proper punctuation
• —Very long sentences without punctuation may still be truncated
• —Running on free Spaces infrastructure with limited computational resources

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference