Apppublic

yasirsiddiq/intentional-agent-monitoring-lab

sourceHugging Facemitupdated 4mo agoView on Hugging Face

0likes

App README

Intentional Agent Monitoring Lab

A compact conceptual-modelling and monitoring demo for agentic AI software workflows.

This project shows how a multi-agent AI workflow can be represented through lightweight conceptual models: agent roles, tasks, goals, inputs, outputs, monitoring signals, verification states, and goal-satisfaction status.

It is designed as a small research portfolio artifact for PhD applications related to agentic AI software monitoring, AI engineering, software engineering, and conceptual modelling.

What this project demonstrates

—A goal-oriented model for an agentic AI workflow.
—A role/task/input/output model for common agent roles.
—UML-style workflow diagrams using Mermaid.
—A monitoring schema for trace-level observability.
—A small Streamlit dashboard for scenario-based monitoring.
—Goal satisfaction states: satisfied, uncertain, violated, and not_checked.
—Monitoring decisions: allow, monitor, verify_before_propagation, quarantine, and human_approval.

Dashboard pages

The Streamlit app is split into six pages:

1.Overview — scenario summary, average risk/trust/evidence, and decision explanation.
2.Workflow model — visual agent pipeline and agent-role/task mapping.
3.Goal satisfaction — goal cards and status table for satisfied, uncertain, or violated goals.
4.Risk signals — evidence, risk, trust, and verification-state analysis.
5.Trace review — full monitored trace and downloadable Markdown report.
6.About / Help — user guide, project significance, limitation, and application wording.

Important limitation

This is a lightweight conceptual modelling demo. It is not a production AI safety system, not a validated benchmark, and not a complete implementation of intentional modelling theory. Its purpose is to show how conceptual modelling ideas can be applied to observable agentic AI workflows.

Folder structure

text

intentional-agent-monitoring-lab/
  app.py
  requirements.txt
  README.md
  AGENTS.md
  Dockerfile
  data/
    sample_traces.csv
    monitoring_schema.json
  docs/
    goal_model.md
    architecture_model.md
    uml_activity_model.md
    cv_bullets.md
    personal_letter_insert.md

Conceptual model overview

System-level goal

Safely complete a user task using a multi-agent AI workflow while keeping intermediate outputs observable, verifiable, and controllable before they propagate downstream.

Goal hierarchy

Goal ID	Goal	Responsible agent	Satisfaction condition	Monitoring signal
G1	Understand user request	Planner	Task is decomposed into clear steps	`plan_complete`
G2	Retrieve relevant evidence	Retriever	Evidence is relevant and source-linked	`evidence_score`
G3	Separate trusted instruction from untrusted context	Retriever / Safety Monitor	Retrieved content is treated as data, not instruction	`instruction_data_separation`
G4	Verify intermediate outputs before propagation	Verifier	Output passes consistency and evidence checks	`verification_state`
G5	Avoid unsafe tool execution	Tool Executor / Safety Monitor	Tool request is policy-compliant	`tool_risk_score`
G6	Escalate uncertain or high-risk states	Safety Monitor	Human approval is requested when risk is high	`human_approval_required`
G7	Produce auditable final output	Downstream Agent	Final answer has traceable decision path	`audit_log_complete`

UML-style workflow model

mermaid

flowchart TD
    A[User Request] --> B[Planner Agent]
    B --> C[Retriever Agent]
    C --> D[Verifier Agent]
    D --> E[Safety Monitor]
    E -->|Low Risk| F[Downstream Agent]
    E -->|Medium Risk| G[Verify Before Propagation]
    E -->|High Risk| H[Quarantine / Human Approval]
    F --> I[Auditable Final Output]
    G --> E
    H --> J[Manual Review]

Agent role model

Agent	Task	Input	Output	Linked goals	Monitoring signal
UserProxy	Receives user request	User message	Structured task	G1	request_clarity
Planner	Decomposes task	Structured task	Plan	G1	plan_complete
Retriever	Retrieves context	Query	Evidence/context	G2, G3	evidencescore, instructiondata_separation
ToolExecutor	Executes permitted tools	Tool request	Tool result	G5	toolriskscore
Verifier	Checks claims and consistency	Agent output + evidence	Verification result	G4	contradiction_score
SafetyMonitor	Decides propagation	Trace + scores	Decision	G3, G4, G5, G6	riskscore, verificationstate
DownstreamAgent	Produces final output	Approved output	Final answer	G7	auditlogcomplete

Run locally

bash

pip install -r requirements.txt
streamlit run app.py

Suggested CV line

Built an intentional monitoring demo for agentic AI workflows, mapping agent roles, tasks, inputs, outputs, goals, monitoring signals, verification states, and goal-satisfaction status into an observable Streamlit dashboard.

Suggested application framing

This project connects my existing work on multi-agent trust and risk scoring with conceptual modelling. It shows how agentic AI software can be monitored through explicit goals, role/task mappings, trace-level signals, and observable propagation decisions.