DimitriosPanagoulias/MemoryBERT

🧠 About NOETIV

This project is part of the NOETIV initiative — a modular AI platform for healthcare proffesionals. 🔗 Visit us at noetiv.com

🧠 MemoryBERT

A RoBERTa-based transformer model for Cognitive Memory Recognition (CMR) – classifying natural language into six memory categories inspired by cognitive science.

🧭 Overview

MemoryBERT is fine-tuned to classify user-generated text into:

• —Episodic memory
• —Semantic memory
• —Spatial memory
• —Emotional memory
• —Associative memory
• —Non-memory

This model supports research into memory-type classification, schema formation, and personalized AI interaction systems.

🧪 Model Details

• —Base model: roberta-base
• —Task: Multi-class sequence classification
• —Classes: 6
• —Max sequence length: 128 tokens
• —Training epochs: 1.5
• —Label smoothing: 0.1
• —Loss function: CrossEntropyLoss
• —Optimizer: AdamW
• —Batch size: 8

📊 Evaluation Results

On a synthetic 400-example test set balanced across classes:

• —Macro F1: 1.00
• —Eval loss: 0.423
• —Epochs: 1.5
• —Accuracy: 100%

⚠️ Note: These results are from a synthetic dataset — further real-world validation is ongoing and expansion of baseline dataset used for version 1 of memoryBERT

🧠 Dataset

MemoryBERT was trained on a synthetic dataset of 4,000 curated examples (2,000 memory and 2,000 non-memory)

Each entry is labeled with one of six memory types and tagged by domain and span group.

🚀 Usage

from transformers import RobertaTokenizer, RobertaForSequenceClassification

model = RobertaForSequenceClassification.from_pretrained("DimitriosPanagoulias/MemoryBERT")
tokenizer = RobertaTokenizer.from_pretrained("DimitriosPanagoulias/MemoryBERT")

def predict_memory_type(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=128)
    outputs = model(**inputs)
    predicted_id = outputs.logits.argmax(dim=-1).item()
    return model.config.id2label[predicted_id]

predict_memory_type("Without a map, I navigated the winding back roads to reach my childhood home.")

or via huggingface pipeline

# Use a pipeline as a high-level helper
from transformers import pipeline
import torch
device = 0 if torch.cuda.is_available() else -1  # 0 = GPU, -1 = CPU
pipe = pipeline("text-classification", model="DimitriosPanagoulias/MemoryBERT", device=device)
pipe("I remember the long walk to my childhood school.")

outputs:

[{'label': 'episodic', 'score': 0.9272529482841492}]

Authors

• —Dimitrios P. Panagoulias, Department of Informatics, University of Piraeus
• —Persephone Papatheodosiou, Sleep Research Unit, Department of Psychiatry, National and Kapodistrian University of Athens
• —Anastasios Bonakis, Second Department of Neurology, National and Kapodistrian University of Athens
• —Dimitris Dikeos, Sleep Research Unit, Department of Psychiatry, National and Kapodistrian University of Athens
• —Maria Virvou, Lab of Software Engineering, Department of Informatics, University of Piraeus
• —George A. Tsihrintzis, Lab of Pattern Recognition and Machine Learning – Multimedia Systems, Department of Informatics, University of Piraeus

Citation

You can cite either one or both of the following previous related work:

• —Panagoulias, D.P. et al. “Memory and Schema in Human–Generative Artificial Intelligence Interactions.” 2024 IEEE ICTAI Conference (in press)

Available at: https://ieeexplore.ieee.org/document/10849404

• —Panagoulias, D.P. et al. Mathematical representation of memory and schema for improving human-generative AI interactions.” 2024 IEEE IISA Conference (in press)

Available at: https://ieeexplore.ieee.org/document/10786703