woningwaardering-llama3-8b-4bit-v1

This model is a fine-tuned version of meta-llama/Meta-Llama-3.1-8B on woonstadrotterdam/woningwaarderingen. Inspired by Ed Donner's price model to predict Amazon product prices.

How many points for this dwelling?

This is an apartment from 1992 with 5 rooms of which 2 are bedrooms. Its surface area is 64m² and its outdoor area is 4m². The energy label is A. The property value is €223k.

Points: 153

Model description

Model is trained to predict the woningwaardering points of a dwelling based on a short description of the dwelling.

Intended uses & limitations

This model is intended for educational and research purposes. However, practical use cases can be imagined. For example, estimates can be made for dwellings based on a short description of the dwelling on a real estate website.

Its main limitation is that is has been trained on a fixed format of dwelling descriptions, and may not generalise to other formats. For its other limitations, see the limitations of the dataset it was trained on.

Training and evaluation data

See woonstadrotterdam/woningwaarderingen for the train, validation and test data.

Training procedure

See scripts/training.ipynb

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 1
• seed: 42
• optimizer: Use OptimizerNames.PAGED_ADAMW with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 7

Framework versions

• PEFT 0.14.0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenisers 0.21.1

Evaluation

See scripts/evaluation.ipynb

MAE and MAPE are chosen as the key metrics for evaluation as they are the most easily interpretable metrics for non-data scientists.