Karim shoair commited on
Commit
7c719ac
·
1 Parent(s): ecc7bcb

docs: add new api references

Browse files
docs/api-reference/proxy-rotation.md ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ search:
3
+ exclude: true
4
+ ---
5
+
6
+ # Proxy Rotation
7
+
8
+ The `ProxyRotator` class provides thread-safe proxy rotation for any fetcher or session.
9
+
10
+ You can import it directly like below:
11
+
12
+ ```python
13
+ from scrapling.fetchers import ProxyRotator
14
+ ```
15
+
16
+ ## ::: scrapling.engines.toolbelt.proxy_rotation.ProxyRotator
17
+ handler: python
18
+ :docstring:
docs/api-reference/response.md ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ search:
3
+ exclude: true
4
+ ---
5
+
6
+ # Response Class
7
+
8
+ The `Response` class wraps HTTP responses returned by all fetchers, providing access to status, headers, body, cookies, and a `Selector` for parsing.
9
+
10
+ You can import the `Response` class like below:
11
+
12
+ ```python
13
+ from scrapling.engines.toolbelt.custom import Response
14
+ ```
15
+
16
+ ## ::: scrapling.engines.toolbelt.custom.Response
17
+ handler: python
18
+ :docstring:
docs/api-reference/spiders.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ search:
3
+ exclude: true
4
+ ---
5
+
6
+ # Spider Classes
7
+
8
+ Here's the reference information for the spider framework classes' parameters, attributes, and methods.
9
+
10
+ You can import them directly like below:
11
+
12
+ ```python
13
+ from scrapling.spiders import Spider, Request, CrawlResult, SessionManager, Response
14
+ ```
15
+
16
+ ## ::: scrapling.spiders.Spider
17
+ handler: python
18
+ :docstring:
19
+
20
+ ## ::: scrapling.spiders.Request
21
+ handler: python
22
+ :docstring:
23
+
24
+ ## Result Classes
25
+
26
+ ## ::: scrapling.spiders.result.CrawlResult
27
+ handler: python
28
+ :docstring:
29
+
30
+ ## ::: scrapling.spiders.result.CrawlStats
31
+ handler: python
32
+ :docstring:
33
+
34
+ ## ::: scrapling.spiders.result.ItemList
35
+ handler: python
36
+ :docstring:
37
+
38
+ ## Session Management
39
+
40
+ ## ::: scrapling.spiders.session.SessionManager
41
+ handler: python
42
+ :docstring:
zensical.toml CHANGED
@@ -54,7 +54,10 @@ nav = [
54
  {Selector = "api-reference/selector.md"},
55
  {Fetchers = "api-reference/fetchers.md"},
56
  {"MCP Server" = "api-reference/mcp-server.md"},
57
- {"Custom Types" = "api-reference/custom-types.md"}
 
 
 
58
  ]},
59
  {"Writing your retrieval system" = "development/adaptive_storage_system.md"},
60
  {"Using Scrapling's custom types" = "development/scrapling_custom_types.md"}
 
54
  {Selector = "api-reference/selector.md"},
55
  {Fetchers = "api-reference/fetchers.md"},
56
  {"MCP Server" = "api-reference/mcp-server.md"},
57
+ {"Custom Types" = "api-reference/custom-types.md"},
58
+ {Response = "api-reference/response.md"},
59
+ {Spiders = "api-reference/spiders.md"},
60
+ {"Proxy Rotation" = "api-reference/proxy-rotation.md"}
61
  ]},
62
  {"Writing your retrieval system" = "development/adaptive_storage_system.md"},
63
  {"Using Scrapling's custom types" = "development/scrapling_custom_types.md"}