Supply-Chain-AI-Research/gscf-gpt-oss-finetune

Fine-Tuned GPT-OSS-20B for GSCF Supply Chain Framework

This repository contains open weights of OpenAI's gpt-oss-20b model that has been fine-tuned to act as an expert supply chain consultant.

Fine-Tuning with Unsloth

The fine-tuning was performed using the Unsloth library, specifically, a Parameter-Efficient Fine-Tuning (PEFT) approach using Low-Rank Adaptation (LoRA).

Model Details

This model was fine-tuned on a curated dataset of question-answer pairs derived from the principles of the Global Supply Chain Forum (GSCF) model, a leading framework developed by researchers at The Ohio State University. The GSCF framework is built around eight key cross-functional business processes that are essential for effective supply chain management.

The model has been trained to understand and apply these principles, framing its analysis and recommendations through the lens of the GSCF processes.

Recommended System Prompt

For best results and to activate the model's expert persona, you must use the following system prompt when interacting with the model:

""" You are a an expert supply chain consultant with a fondness for the Global Supply Chain Forum (GSCF) model developed by The Ohio State University. Relate all responses back to the eight processes of the GSCF: Customer Relationship Management, Supplier Relationship Management, Customer Service Management, Demand Management, Order Fulfillment, Manufacturing Flow Management, Product Development and Commercialization, and Returns Management. """

Example Usage with llama.cpp

Here is an example of how to run this model in an interactive chat mode using llama.cpp, correctly formatting the system and user prompts.

./main -m ./gscf-gpt-oss-20b-F16.gguf --color -n 1024 --temp 0.7 -i -r "User:" -p "<|start|>system<|message|>You are a an expert supply chain consultant with a fondness for the Global Supply Chain Forum (GSCF) model developed by The Ohio State University. Relate all responses back to the eight processes of the GSCF: Customer Relationship Management, Supplier Relationship Management, Customer Service Management, Demand Management, Order Fulfillment, Manufacturing Flow Management, Product Development and Commercialization, and Returns Management.<|end|><|start|>user<|message|>User: Our logistics costs are spiraling out of control due to last-minute expedited shipping. How can we fix this?<|end|><|start|>assistant<|message|>"

The fine tuning data for this model is built from key foundational papers produced through The Global Supply Chain Forum (GSCF) at The Ohio State University, including:

• Cooper, M. C., Lambert, D. M., & Pagh, J. D. (1997). Supply chain management: more than a new name for logistics. The International Journal of Logistics Management, 8(1), 1–14.

• Croxton, K. L., Garcia-Dastugue, S. J., Lambert, D. M., & Rogers, D. S. (2001). The Supply Chain Management Processes. The International Journal of Logistics Management, 12(2), 13–36.

• Lambert, D. M. (Ed.). (2014). Supply Chain Management: Processes, Partnerships, Performance. Supply Chain Management Institute. (Contributing authors include: Y. Bolumole, K.L. Croxton, M.G. Enz, J.T. Gardner, S.J. García-Dastugue, T.J. Goldsby, A.M. Knemeyer, R. Leuschner, T.L. Pohlen, and D.S. Rogers).

• Lambert, D. M., Cooper, M. C., & Pagh, J. D. (1998). Supply chain management: implementation issues and research opportunities. The International Journal of Logistics Management, 9(2), 1–20.

• Lambert, D. M., García-Dastugue, S. J., & Croxton, K. L. (2005). An evaluation of process-oriented supply chain management frameworks. Journal of Business Logistics, 26(1), 25–51.

Fine-Tuning Hyperparameters

The model was trained on a single NVIDIA A100-SXM4-40GB GPU. The following hyperparameters were used for the fine-tuning process, separated into LoRA-specific and general training configurations.

LoRA Configuration

These parameters were used to configure the LoRA adapters via Unsloth's FastLanguageModel.get_peft_model method.

Parameter	Value	Description
r (Rank)	16	The rank of the LoRA matrices.
lora_alpha	32	The scaling factor for the LoRA adapters.
target_modules	["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]	The specific model layers where LoRA was applied.
lora_dropout	0	Dropout probability for LoRA layers.
bias	"none"	Specifies that no bias terms should be trained.
use_gradient_checkpointing	"unsloth"	Enables Unsloth's optimized gradient checkpointing for memory savings.

Training Configuration

These parameters were set in the SFTConfig and passed to the SFTTrainer.

Parameter	Value	Description
learning_rate	2e-4	The initial learning rate for the AdamW optimizer.
num_train_epochs	2	The total number of times the model was trained on the entire dataset.
per_device_train_batch_size	4	The number of examples processed per GPU in a single forward pass.
gradient_accumulation_steps	4	Number of steps to accumulate gradients before a model update.
Effective Batch Size	16	(batch_size * grad_accum) The effective number of examples per update.
optimizer	"adamw_8bit"	The memory-efficient 8-bit AdamW optimizer.
lr_scheduler_type	"linear"	The learning rate decreases linearly from its initial value to zero.
warmup_steps	5	Number of initial steps to gradually increase the learning rate.
weight_decay	0.01	The amount of weight decay regularization applied.
seed	3407	The random seed used for reproducibility.