Upload folder using huggingface_hub

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 xiu xiu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,188 @@
+---
+license: mit
+language:
+- zh
+- en
+datasets:
+- IowaCat/page_type_inference_dataset
+metrics:
+- accuracy: 0.99
+pipeline_tag: text-generation
+tags:
+- url-classification
+- list-page-detection
+- detail-page-detection
+- qwen
+- fine-tuning
+- lora
+- url-parser
+widget:
+- text: "https://example.com/product/12345"
+- text: "https://example.com/category/electronics"
+---
+
+# URL Page Type Classifier
+
+<div align="center">
+
+![Model Size](https://img.shields.io/badge/Model%20Size-1.5B-blue)
+![License](https://img.shields.io/badge/License-MIT-green)
+![Accuracy](https://img.shields.io/badge/Accuracy-99%25-green)
+
+</div>
+
+## 📋 概述
+
+基于 Qwen2.5-1.5B + LoRA 微调的URL类型分类模型，用于判断URL是列表页还是详情页。
+
+## 🏗️ 模型架构
+
+| 项目 | 详情 |
+|------|------|
+| **基础模型** | Qwen/Qwen2.5-1.5B |
+| **微调方法** | LoRA (r=16, alpha=32) |
+| **参数量** | 1.5B |
+| **可训练参数** | ~18M (1.18%) |
+
+## 📊 训练数据
+
+- **数据集**: IowaCat/page_type_inference_dataset
+- **训练样本**: 10,000条URL (5000列表页 + 5000详情页)
+- **数据来源**: HuggingFace Datasets
+
+### 数据分布
+
+| 类型 | 数量 | 比例 |
+|------|------|------|
+| 列表页 (List Page) | 5,000 | 50% |
+| 详情页 (Detail Page) | 5,000 | 50% |
+
+## ⚙️ 训练配置
+
+```python
+{
+    "base_model": "Qwen/Qwen2.5-1.5B",
+    "lora_rank": 16,
+    "lora_alpha": 32,
+    "lora_dropout": 0.05,
+    "num_train_epochs": 3,
+    "per_device_train_batch_size": 2,
+    "gradient_accumulation_steps": 8,
+    "learning_rate": 2e-4,
+    "fp16": true,
+    "optimizer": "adamw_torch",
+    "lr_scheduler_type": "cosine"
+}
+```
+
+## 📈 性能评估
+
+### 测试结果
+
+| 测试集 | 样本数 | 准确率 |
+|--------|--------|--------|
+| 验证集 | 100 | **99%** |
+
+### 示例预测
+
+| URL | 预测结果 |
+|-----|----------|
+| `https://example.com/products/category` | 列表页 (List Page) |
+| `https://example.com/product/12345` | 详情页 (Detail Page) |
+| `https://example.com/search?q=test` | 列表页 (List Page) |
+| `https://example.com/item/abc123` | 详情页 (Detail Page) |
+| `https://example.com/list/all` | 列表页 (List Page) |
+
+## 🚀 快速开始
+
+### 安装依赖
+
+```bash
+pip install transformers peft torch
+```
+
+### 推理代码
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "windlx/url-classifier-model"
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
+
+# 要分类的URL
+url = "https://example.com/product/12345"
+
+# 构建提示
+prompt = f"""请判断以下URL是列表页还是详情页。
+
+URL: {url}
+类型: """
+
+# 推理
+inputs = tokenizer(prompt, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=10, do_sample=False)
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+# 提取结果
+if "详情页" in response or "Detail Page" in response:
+    result = "详情页 (Detail Page)"
+else:
+    result = "列表页 (List Page)"
+
+print(f"URL: {url}")
+print(f"类型: {result}")
+```
+
+### 使用 GPU
+
+```python
+# 自动使用GPU
+model = AutoModelForCausalLM.from_pretrained(
+    model_name, 
+    trust_remote_code=True,
+    device_map="auto",
+    torch_dtype="auto"
+)
+```
+
+### 使用 CPU
+
+```python
+# 强制使用CPU
+model = AutoModelForCausalLM.from_pretrained(
+    model_name, 
+    trust_remote_code=True,
+    device_map="cpu",
+    torch_dtype="float32"
+)
+```
+
+## ⚠️ 局限性
+
+1. **仅基于URL字符串** - 不访问实际网页内容
+2. **依赖URL路径规范** - 对于URL路径不规范的网站，准确率可能较低
+3. **仅支持中英文** - 主要针对中文URL优化
+
+## 📝 使用场景
+
+- 🔍 **搜索引擎优化 (SEO)** - 识别网站页面结构
+- 🕷️ **网页爬虫** - 判断链接类型，优化爬取策略
+- 📊 **网站分析** - 统计列表页和详情页比例
+- 🔗 **链接分类** - 大规模URL分类处理
+
+## 📁 相关链接
+
+- **GitHub仓库**: https://github.com/xiuxiu/url-classifier
+- **HuggingFace模型**: https://huggingface.co/windlx/url-classifier-model
+- **训练数据集**: https://huggingface.co/datasets/IowaCat/page_type_inference_dataset
+
+## 🙏 致谢
+
+- [Qwen](https://github.com/QwenLM/Qwen2) - 提供基础模型
+- [LoRA](https://github.com/microsoft/LoRA) - 高效微调方法
+- [HuggingFace](https://huggingface.co/) - 模型托管平台
+
+## 📄 许可
+
+[LICENSE](LICENSE) - MIT License