add README

README.md

@@ -0,0 +1,146 @@
+---
+language:
+- en
+base_model:
+- google-bert/bert-base-uncased
+pipeline_tag: token-classification
+library_name: transformers
+tags:
+- token-classification
+- named-entity-recognition
+- ner
+- bert
+- logs
+datasets:
+- Aliph0th/logtheus-ml-ds
+---
+# Log Entity Extractor (BERT-based Token Classifier)
+
+A fine-tuned BERT model for extracting canonical attributes from log lines using token classification (NER-style task). Created for my course work
+
+## Model Description
+
+This model is based on `bert-base-uncased` and trained to perform **token classification** on log messages. It extracts structured attributes (service, level, event, error_code, user_id, ip, etc.) from unstructured log text using a BIO tagging scheme.
+
+**Use case:** Convert raw log lines into canonical, structured key-value pairs for downstream analysis, alerting, or aggregation.
+
+## Model Details
+
+- **Base Model:** `bert-large-uncased`
+- **Task:** Token Classification (Named Entity Recognition)
+- **Training Data:** Annotated log lines (character-level entity offsets)
+- **Input:** Raw log text (string)
+- **Output:** Per-token BIO labels → grouped entities as canonical attributes
+
+## Canonical Label Set
+
+The model extracts attributes from these canonical fields:
+
+| Field | Description |
+|-------|-------------|
+| service | Application or service name (e.g., "auth", "api") |
+| level | Log level (e.g., "info", "error", "warn") |
+| timestamp | Timestamp or date reference |
+| environment | Deployment environment (e.g., "prod", "staging") |
+| event | Event type or action (e.g., "login", "request") |
+| error_message | Human-readable error message |
+| status_code | HTTP or service status code |
+| duration | Duration |
+| ip | IP address (client or server) |
+| method | HTTP method (GET, POST, etc.) |
+| path | URL path or resource path |
+| useragent | User-Agent header |
+| hostname | Server hostname |
+
+## Usage
+
+### Installation
+
+```bash
+pip install transformers torch
+```
+
+### Python (Hugging Face Transformers)
+
+```python
+from transformers import AutoTokenizer, AutoModelForTokenClassification
+import torch
+model_name = "Aliph0th/logtheus-ml"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForTokenClassification.from_pretrained(model_name)
+text = "[auth] failed login for user 123 from 10.1.2.3 code=E401"
+# Tokenize and forward pass
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
+outputs = model(**inputs)
+logits = outputs.logits
+# Get predicted label IDs
+predicted_ids = torch.argmax(logits, dim=-1)
+# Map back to label names
+id2label = model.config.id2label
+predictions = [[id2label[int(p)] for p in pred] for pred in predicted_ids]
+print(predictions)
+```
+
+This returns a structured JSON object with:
+- `attributes`: High-confidence extractions (dict of canonical_field → value)
+- `low_confidence_attributes`: Below-threshold extractions
+- `attribute_confidence`: Per-field confidence scores
+- `message`: Original log text
+- `confidence`: Overall prediction confidence (0-1)
+- `model_version`: Model version string
+
+## Training
+
+### Dataset Format
+
+Training data in JSONL format with character-offset annotations:
+
+```json
+{"id":"1","text":"[auth] failed login for user 123 from 10.1.2.3","entities":[{"start":1,"end":5,"label":"service"},{"start":28,"end":32,"label":"user_id"},{"start":38,"end":46,"label":"ip"}]}
+```
+
+Used dataset -[Aliph0th/logtheus-ml-ds](https://huggingface.co/datasets/Aliph0th/logtheus-ml-ds)
+
+**Fields:**
+- `text`: Raw log line (string)
+- `entities`: List of entity annotations
+  - `start`, `end`: Character-level offsets in text (0-indexed)
+  - `label`: Canonical field name
+
+### Training Procedure
+
+```bash
+# 1. Prepare raw log files (deduplicate, split train/val)
+python scripts/process_data.py data/annotated/ --p 0.8
+# 2. Train model
+python training/train_token_classifier.py \
+  --train-file data/train.jsonl \
+  --val-file data/val.jsonl \
+  --output-dir artifacts/model_v1 \
+  --base-model bert-base-uncased \
+  --epochs 5 \
+  --batch-size 16
+```
+
+**Hyperparameters:**
+- Learning rate: 3e-5
+- Batch size: 16 (per device)
+- Epochs: 5 (with early stopping by F1)
+- Optimizer: AdamW
+- Weight decay: 0.01
+
+## Limitations
+
+- **English logs only:** Trained on ASCII/UTF-8 log text in English
+- **Format dependency:** Works best on semi-structured logs (key=value, JSON, or naturally worded messages); highly custom formats may need domain-specific preprocessing
+
+## Contact & Support
+
+For issues, questions, or contributions, please visit:
+- **Repository:** https://github.com/Aliph0th/logtheus-ml
+- **Issues:** https://github.com/Aliph0th/logtheus-ml/issues
+
+## Acknowledgments
+
+- Based on [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805)
+- Built with [Hugging Face Transformers](https://huggingface.co/transformers/)