Apppublic

IntuiAI/intelligent-pid

sourceHugging Facemitupdated 2y agoView on Hugging Face

0likes

App README

P&ID Processing with AI-Powered Graph Construction

Overview

This project processes P&ID (Piping and Instrumentation Diagram) images using multiple AI models for symbol detection, text recognition, and line detection. It constructs a graph representation of the diagram and provides an interactive interface for querying the diagram's contents.

Process Flow

mermaid

graph TD
    subgraph "Document Input"
        A[Upload Document] --> B[Validate File]
        B -->|PDF/Image| C[Document Processor]
        B -->|Invalid| ERR[Error Message]
        C -->|PDF| D1[Extract Pages]
        C -->|Image| D2[Direct Process]
    end

    subgraph "Image Preprocessing"
        D1 --> E[Optimize Image]
        D2 --> E
        E -->|CLAHE Enhancement| E1[Contrast Enhancement]
        E1 -->|Denoising| E2[Clean Image]
        E2 -->|Binarization| E3[Binary Image]
        E3 -->|Resize| E4[Normalized Image]
    end

    subgraph "Line Detection Pipeline"
        E4 --> L1[Load DeepLSD Model]
        L1 --> L2[Scale Image 0.1x]
        L2 --> L3[Grayscale Conversion]
        L3 --> L4[Model Inference]
        L4 --> L5[Scale Coordinates]
        L5 --> L6[Draw Lines]
    end

    subgraph "Detection Pipeline"
        E4 --> F[Symbol Detection]
        E4 --> G[Text Detection]
        
        F --> S1[Load YOLO Models]
        G --> T1[Load OCR Models]
        
        S1 --> S2[Detect Symbols]
        T1 --> T2[Detect Text]
        
        S2 --> S3[Process Symbols]
        T2 --> T3[Process Text]
        
        L6 --> L7[Process Lines]
    end

    subgraph "Data Integration"
        S3 --> I[Data Aggregation]
        T3 --> I
        L7 --> I
        I --> J[Create Edges]
        J --> K[Build Graph Network]
        K --> L[Generate Knowledge Graph]
    end

    subgraph "User Interface"
        L --> M[Interactive Visualization]
        M --> N[Chat Interface]
        N --> O[Query Processing]
        O --> P[Response Generation]
        P --> N
    end

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style F fill:#fbb,stroke:#333,stroke-width:2px
    style G fill:#bfb,stroke:#333,stroke-width:2px
    style H fill:#bbf,stroke:#333,stroke-width:2px
    style I fill:#fbf,stroke:#333,stroke-width:2px
    style N fill:#bbf,stroke:#333,stroke-width:2px
    
    %% Add style for model nodes
    style SM1 fill:#ffe6e6,stroke:#333,stroke-width:2px
    style SM2 fill:#ffe6e6,stroke:#333,stroke-width:2px
    style LM1 fill:#e6e6ff,stroke:#333,stroke-width:2px
    style DC1 fill:#e6ffe6,stroke:#333,stroke-width:2px
    style DC2 fill:#e6ffe6,stroke:#333,stroke-width:2px

Architecture

[image]

Features

—Multi-modal AI Processing:
—Combined OCR approach using Tesseract, EasyOCR, and DocTR
—Symbol detection with optimized thresholds
—Intelligent line and connection detection
—Document Processing:
—Support for PDF, PNG, JPG, JPEG formats
—Automatic page extraction from PDFs
—Image optimization pipeline
—Text Detection Types:
—Equipment Tags
—Line Numbers
—Instrument Tags
—Valve Numbers
—Pipe Sizes
—Flow Directions
—Service Descriptions
—Process Instruments
—Nozzles
—Pipe Connectors
—Data Integration:
—Automatic edge detection
—Relationship mapping
—Confidence scoring
—Detailed detection statistics
—User Interface:
—Interactive visualization tabs
—Real-time processing feedback
—AI-powered chat interface
—Knowledge graph exploration

The entire process is visualized through an interactive Gradio-based UI, allowing users to upload a P&ID image, follow the detection steps, and view both the results and insights in real time.

Key Files

—gradioChatApp.py: The main Gradio app script that handles the frontend and orchestrates the overall flow.
—symbol_detection.py: Module for detecting symbols using YOLO models.
—text_detection_combined.py: Unified module for text detection using multiple OCR engines (Tesseract, EasyOCR, DocTR).
—line_detection_ai.py: Module for detecting lines and connections using AI.
—data_aggregation.py: Aggregates detected elements into a structured format.
—graph_construction.py: Constructs the graph network from aggregated data.
—graph_processor.py: Handles graph visualization and processing.
—pdf_processor.py: Handles PDF document processing and page extraction.

Setup and Installation

1.Clone the repository:

bash

git clone https://github.com/IntuigenceAI/intui-PnID-POC.git
cd intui-PnID-POC

1.Install dependencies using uv:

bash

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv pip install -r requirements.txt

1.Download required models:

bash

python download_model.py  # Downloads DeepLSD model for line detection

1.Run the application:

bash

python gradioChatApp.py

Models

Line Detection Model

—DeepLSD Model:
—File: deeplsd_md.tar
—Purpose: Line segment detection in P&ID diagrams
—Input Resolution: Variable (scaled to 0.1x for performance)
—Processing: Grayscale conversion and binary thresholding

Text Detection Models

—Combined OCR Approach:
—Tesseract OCR
—EasyOCR
—DocTR
—Purpose: Text recognition and classification

Graph Processing

—NetworkX-based:
—Purpose: Graph construction and analysis
—Features: Node linking, edge creation, path analysis

Updating the Environment

To update the environment, use the following:

bash

conda env update --file environment.yml --prune

This command will update the environment according to changes made in the environment.yml.

Step 6: Deactivate the environment

When you're done, deactivate the environment by:

bash

conda deactivate

1.Upload a P&ID image through the interface.
2.Follow the sequential steps of symbol, text, and line detection.
3.View the generated graph and AI agent's reasoning in the real-time chat box.
4.Save and export the results if satisfactory.

Folder Structure

├── assets/
│   └── AiAgent.png
│   └── llm.png
├── gradioApp.py
├── symbol_detection.py
├── text_detection_combined.py
├── line_detection_ai.py
├── data_aggregation.py
├── graph_construction.py
├── graph_processor.py
├── pdf_processor.py
├── pnid_agent.py
├── requirements.txt
├── results/
├── models/
│   └── symbol_detection_model.pth

/models Folder

—models/symbol_detection_model.pth: This folder contains the pre-trained model for symbol detection in P&ID diagrams. This model is crucial for detecting key symbols such as valves, instruments, and pipes in the diagram. Make sure to download the model and place it in the /models directory before running the app.

Future Work

—Advanced Symbol Recognition: Improve symbol detection by integrating more sophisticated recognition models.
—Graph Enhancement: Introduce more complex graph structures and logic for representing the relationships between the diagram's elements.
—Data Export: Allow export in additional formats such as DEXPI-compliant XML or JSON.

Docker Information

We'll cover the basic docker operations here.

Building

There is a dockerfile for each different project (they have slightly different requiremnts).

`gradioChatApp.py`

Run this one as follows:

> docker build -t exp-pnid-to-graph_chat-w-graph:0.0.4 -f Dockerfile-chatApp .
> docker tag exp-pnid-to-graph_chat-w-graph:0.0.4 intaicr.azurecr.io/intai/exp-pnid-to-graph_chat-w-graph:0.0.4

Deploying to ACR

`gradioChatApp.py`

> az login
> az acr login --name intaicr
> docker push intaicr.azurecr.io/intai/exp-pnid-to-graph_chat-w-graph:0.0.4

Models

Symbol Detection Models

—Intui_SDM_41.pt: Primary model for equipment and large symbol detection
—Classes: Equipment, Vessels, Heat Exchangers
—Input Resolution: 1280x1280
—Confidence Threshold: 0.3-0.7 (adaptive)

—Intui_SDM_20.pt: Secondary model for instrument and small symbol detection
—Classes: Instruments, Valves, Indicators
—Input Resolution: 1280x1280
—Confidence Threshold: 0.3-0.7 (adaptive)

Line Detection Model

—intui_LDM_01.pt: Specialized model for line and connection detection
—Classes: Solid Lines, Dashed Lines
—Input Resolution: 1280x1280
—Confidence Threshold: 0.5

Text Detection Models

—Tesseract: v5.3.0
—Configuration:
—OEM Mode: 3 (Default)
—PSM Mode: 11 (Sparse text)
—Custom Whitelist: A-Z, 0-9, special characters

—EasyOCR: v1.7.1
—Configuration:
—Language: English
—Paragraph Mode: False
—Height Threshold: 2.0
—Width Threshold: 2.0
—Contrast Threshold: 0.2

—DocTR: v0.6.0
—Models:
—fast_base-688a8b34.pt
—crnnvgg16bn-9762b0b0.pt

P&ID Line Detection

A deep learning-based pipeline for detecting lines in P&ID diagrams using DeepLSD.

Architecture

mermaid

graph TD
    A[Input Image] --> B[Line Detection]
    B --> C[DeepLSD Model]
    C --> D[Post-processing]
    D --> E[Output JSON/Image]
    
    subgraph Line Detection Pipeline
    B --> F[Image Preprocessing]
    F --> G[Scale Image 0.1x]
    G --> H[Grayscale Conversion]
    H --> C
    C --> I[Scale Coordinates]
    I --> J[Draw Lines]
    J --> E
    end

Setup

Prerequisites

—Python 3.12+
—uv (for dependency management)
—Git
—CUDA-capable GPU (optional)

Installation

1.Clone the repository:

bash

git clone https://github.com/IntuigenceAI/intui-PnID-POC.git
cd intui-PnID-POC

1.Install dependencies using uv:

bash

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv pip install -r requirements.txt

1.Download DeepLSD model:

bash

python download_model.py

Usage

1.Run the line detection:

bash

python line_detection_ai.py

The script will:

—Load the DeepLSD model
—Process input images at 0.1x scale for performance
—Generate line detections
—Save results as JSON and annotated images

Configuration

Key parameters in line_detection_ai.py:

—scale_factor: Image scaling (default: 0.1)
—device: CPU/GPU selection
—mask_json_paths: Paths to text/symbol detection results

Input/Output

Input

—Original P&ID images
—Optional text/symbol detection JSON files for masking

Output

—Annotated images with detected lines
—JSON files containing line coordinates and metadata

Project Structure

├── line_detection_ai.py    # Main line detection script
├── detectors.py           # Line detector implementation
├── download_model.py      # Model download utility
├── models/               # Directory for model files
│   └── deeplsd_md.tar   # DeepLSD model weights
├── results/              # Output directory
└── requirements.txt      # Project dependencies

Dependencies

Key dependencies:

—torch
—opencv-python
—numpy
—DeepLSD

See requirements.txt for the complete list.

Contributing

1.Fork the repository
2.Create your feature branch (git checkout -b feature/amazing-feature)
3.Commit your changes (git commit -m 'Add some amazing feature')
4.Push to the branch (git push origin feature/amazing-feature)
5.Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

—DeepLSD for the line detection model
—Original P&ID processing pipeline by IntuigenceAI

title: PnID Diagram Analyzer emoji: 🔍 colorFrom: blue colorTo: red sdk: gradio sdkversion: 4.19.2 appfile: gradioChatApp.py pinned: false ---

PnID Diagram Analyzer

This app analyzes PnID diagrams using AI to detect and interpret various elements.

Features

—Line detection
—Symbol recognition
—Text detection
—Graph construction