UmerKhan261/agent-readiness-gemma4-e2b-lora

Agent Readiness Optimizer

Fine-tuned Gemma 4 E2B that turns raw product listings into structured JSON for AI shopping agents. Built for the Gemma 4 Good Hackathon (Digital Equity track).

The problem

AI shopping agents (ChatGPT, Gemini, Perplexity) can only discover products with structured data following the Agentic Commerce Protocol or Universal Commerce Protocol. Most small merchants don't have the technical resources to produce this data. Their products are invisible to the next generation of commerce.

This model closes that gap. Feed it a raw product page, get back a 15-field JSON listing ready for ACP/UCP.

What it does

Input: raw product data (title, URL, scraped HTML)

Output: structured JSON with 15 fields:

{
  "title": "Vitamin C Face Cream",
  "product_url": "https://www.100percentpure.com/products/vitamin-c-face-cream",
  "availability": "in_stock",
  "condition": "new",
  "brand": {"name": "100% PURE"},
  "price": {"amount": "55.00", "currency": "USD"},
  "description": "100% Pure is the Vitamin C expert. We first discovered and patented a way to stabilize Vitamin C from oxidizing in skincare in 1999. Concentrated with Tetrahexyldecyl Ascorbate, the gold standard for the highest-quality and most stable form of Vitamin C...",
  "image_url": "https://cdn.shopify.com/s/files/1/0648/1955/files/Vitamin_C_Face_Cream.png",
  "additional_images": ["https://cdn.shopify.com/.../VitaminCFaceCream_PB.png", "..."],
  "categorization": {"product_type": "Skin Care", "google_product_category": 3090},
  "attributes": {"color": "Cream", "age_group": "adult"},
  "identifiers": {"sku": "1SVCFC1.4OZ"},
  "identifier_exists": false,
  "shipping": {"free_shipping": false, "base_currency": "USD", "estimated_days": 5},
  "returns": {"returnable": true, "window_days": 60, "return_policy_country": "US"}
}

The base Gemma 4 E2B produces JSON with invented field names (brand_name, category_id, gtin, mpn) that don't match any commerce protocol. It never outputs availability, product_url, categorization, shipping, or returns. This fine-tuned model produces the exact schema.

Results

Evaluated on 5 held-out products, base model vs fine-tuned:

	Base Gemma 4 E2B	Fine-tuned
Correct schema (15 fields)	0/5	3/5
Has categorization	0/5	3/5
Has shipping + returns	0/5	3/5
Has SEO fields (3+/4)	1/5	3/5
Field exact match rate	38%	71%
Avg description length	113 chars	258 chars

The base model always produces valid JSON but with wrong field names. The fine-tuned model produces the correct schema when it succeeds, but 2/5 samples failed JSON parsing (truncated output). Training for more epochs would improve this.

Honest assessment: 60% valid output rate after 1 epoch. Not production-ready. Needs 2-3 epochs and longer generation length to reach 80%+.

Training

Base model	google/gemma-4-E2B-it
Method	QLoRA (4-bit NF4, double quant)
LoRA	r=16, alpha=16, dropout=0.05
Target modules	q, k, v, o, gate, up, down projections
Trainable params	29.9M / 3.7B (0.80%)
Training data	22,623 SFT pairs from 11 online stores
Steps	1,500 (1.06 epochs)
Effective batch	16 (batch=2, grad_accum=8)
LR	2e-4, cosine decay
Optimizer	paged_adamw_8bit
Precision	bf16
GPU	NVIDIA L4 24GB (Lightning AI)
Time	~7 hours
Best loss	1.57 (step 1210)
Final loss	1.78

Dataset

22,623 product enrichment pairs extracted from 11 real stores using shopextract. 1,191 held out for eval.

Top categories in training data:

Category	Count
Womens	2,732
Mens	2,122
Accessories	1,313
Wovens	1,004
Dress	757
Shoes	749
Knits	691
Tops	495
Outerwear	488

Mostly fashion/apparel. Performance on electronics, home goods, or food products is untested.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model = AutoModelForCausalLM.from_pretrained(
    "UmerKhan261/agent-readiness-gemma4-e2b-lora",
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True, bnb_4bit_use_double_quant=True,
        bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16,
    ),
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("UmerKhan261/agent-readiness-gemma4-e2b-lora")

prompt = "You are a product data enrichment agent. Given an incomplete product listing, enrich it with missing attributes and output JSON conforming to the EnrichedProduct schema.\n\nTitle: Organic Green Tea\nURL: https://example.com/green-tea\nPrice: $12.99"

messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_tensors="pt", return_dict=True,
)
with torch.no_grad():
    out = model.generate(inputs["input_ids"].to("cuda"), max_new_tokens=1024, temperature=0.3)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Note: Gemma 4 uses Gemma4ClippableLinear which PEFT doesn't recognize. If loading as base + adapter, you need the monkey-patch before calling from_pretrained.

Limitations

• 60% valid JSON rate. 40% of outputs fail to parse, likely due to truncation.
• Trained for 1 epoch only. More epochs would help.
• English only. No multilingual support.
• Fashion/apparel heavy. Untested on electronics, food, home goods.
• No image understanding used during training (text-only SFT).
• Not production-ready without post-processing and validation.

License

Gemma license

Author

Muhammad Umer Khan — GitHub | HuggingFace