gitea/geziyor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Caching policies added.

Browse Source 
We used httpcache library to implement this. As it was not possible to support different policies, I mostly copied and modified it.
This commit is contained in:
Musab Gültekin
2019-07-07 12:18:40 +03:00
parent 0d6c2a6864
commit 90d2be2210
13 changed files with 2349 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -8,7 +8,7 @@ Geziyor is a blazing fast web crawling and web scraping framework. It can be use
## Features
- 5.000+ Requests/Sec
- JS Rendering
- Caching (Memory/Disk)
- Caching (Memory/Disk/LevelDB)
- Automatic Data Exporting (JSON, CSV, or custom)
- Metrics (Prometheus, Expvar, or custom)
- Limit Concurrency (Global/Per Domain)