Update README.md
Browse files
README.md
CHANGED
|
@@ -11,6 +11,10 @@ pinned: false
|
|
| 11 |
|
| 12 |
This organization is dedicated to the development of high-performance natural language processing (NLP) architectures for the major and regional languages of the Philippines. Our objective is to bridge the digital divide for low-resource languages through state-of-the-art model alignment, knowledge distillation, and the deployment of efficient, edge-ready AI models.
|
| 13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
## Technical Roadmap
|
| 15 |
|
| 16 |
### Phase 1: Foundation Model Alignment and NMT Parity
|
|
@@ -28,6 +32,7 @@ This organization is dedicated to the development of high-performance natural la
|
|
| 28 |
* **Technical Detail:** Transitioning from standard Transformers to LFM 2.5 allows for linear scaling and reduced memory footprints. We use the distilled datasets from Phase 2 to train "Student" models that replicate the output distribution of the larger Llama models. Final optimization includes Direct Preference Optimization (DPO) to refine cultural and grammatical nuance for each specific language.
|
| 29 |
* **Milestone:** A suite of specialized, deployment-ready models (1.2B to 3B parameters) optimized for edge computing and local hardware integration.
|
| 30 |
|
|
|
|
| 31 |
---
|
| 32 |
|
| 33 |
## Stakeholder Engagement and Collaboration
|
|
|
|
| 11 |
|
| 12 |
This organization is dedicated to the development of high-performance natural language processing (NLP) architectures for the major and regional languages of the Philippines. Our objective is to bridge the digital divide for low-resource languages through state-of-the-art model alignment, knowledge distillation, and the deployment of efficient, edge-ready AI models.
|
| 13 |
|
| 14 |
+
|
| 15 |
+
<details>
|
| 16 |
+
<summary>Click to view our Technical Roadmap </summary>
|
| 17 |
+
|
| 18 |
## Technical Roadmap
|
| 19 |
|
| 20 |
### Phase 1: Foundation Model Alignment and NMT Parity
|
|
|
|
| 32 |
* **Technical Detail:** Transitioning from standard Transformers to LFM 2.5 allows for linear scaling and reduced memory footprints. We use the distilled datasets from Phase 2 to train "Student" models that replicate the output distribution of the larger Llama models. Final optimization includes Direct Preference Optimization (DPO) to refine cultural and grammatical nuance for each specific language.
|
| 33 |
* **Milestone:** A suite of specialized, deployment-ready models (1.2B to 3B parameters) optimized for edge computing and local hardware integration.
|
| 34 |
|
| 35 |
+
</details>
|
| 36 |
---
|
| 37 |
|
| 38 |
## Stakeholder Engagement and Collaboration
|