Musab Gültekin
|
76a687e193
|
Do function refactored
|
2019-06-13 20:26:07 +03:00 |
|
Musab Gültekin
|
8a6e19a031
|
New requests on StartRequests func will be made using Geziyor's methods. Not Requests chan
Options field exported.
|
2019-06-13 14:06:37 +03:00 |
|
Musab Gültekin
|
d56ea161a5
|
Making new requests on StartRequestsFunc is simplified by using channels
|
2019-06-12 21:54:57 +03:00 |
|
Musab Gültekin
|
f7f4e401e2
|
Metadata adding on requests support added. StartRequests function implemented.
|
2019-06-12 21:30:45 +03:00 |
|
Musab Gültekin
|
bd8d58576f
|
Start requests function implemented.
|
2019-06-12 12:40:38 +03:00 |
|
Musab Gültekin
|
2f6cb06982
|
Disabling charset detection implemented.
|
2019-06-12 11:44:31 +03:00 |
|
Musab Gültekin
|
a311a0f998
|
CSV exporter support added. Not finished for map type.
|
2019-06-11 20:42:22 +03:00 |
|
Musab Gültekin
|
bbdc3bcacd
|
Exporters made optional, as some scrapers only want to see data in console.
|
2019-06-11 18:59:37 +03:00 |
|
Musab Gültekin
|
b8305d5e1a
|
Limiting body reading support implemented.
|
2019-06-11 16:19:30 +03:00 |
|
Musab Gültekin
|
3790295658
|
Multiple Exporters and custom Exporters support added.
|
2019-06-11 16:10:49 +03:00 |
|
Musab Gültekin
|
e4e8723426
|
Callback are now mandatory as almost all the scrapers use it.
|
2019-06-11 14:24:48 +03:00 |
|
Musab Gültekin
|
ca2414c5c8
|
Request callbacks added.
Recover from all panics and continue scraping.
Only parse HTML if response is HTML.
|
2019-06-09 21:13:30 +03:00 |
|
Musab Gültekin
|
7abc7a370d
|
Disabling logs support added.
|
2019-06-09 19:14:46 +03:00 |
|
Musab Gültekin
|
b973c1c064
|
Request delays support added
|
2019-06-09 14:24:53 +03:00 |
|
Musab Gültekin
|
2263108838
|
User-Agent change support added.
|
2019-06-09 13:43:17 +03:00 |
|
Musab Gültekin
|
9263877339
|
Exporting data all types support added.
|
2019-06-09 13:22:20 +03:00 |
|
Musab Gültekin
|
d967555b62
|
Global and Domain Concurrency limit implemented. Updated README
|
2019-06-09 11:53:40 +03:00 |
|
Musab Gültekin
|
a9aaf86df3
|
Automatic determining response and decoding it.
|
2019-06-09 10:46:32 +03:00 |
|
Musab Gültekin
|
f2fba85266
|
Timeout support added
|
2019-06-08 20:36:43 +03:00 |
|
Musab Gültekin
|
2e3bd18430
|
Options refactored to its own file. Timeout increased to 60 sec
|
2019-06-08 20:36:43 +03:00 |
|
Musab Gültekin
|
b90908066b
|
Head API added. Opt renamed to Options. Tests updated. More documentation added.
|
2019-06-08 20:36:43 +03:00 |
|
Musab Gültekin
|
815ae7eec5
|
Do request support added. Updated docs.
|
2019-06-08 19:45:48 +03:00 |
|
Musab Gültekin
|
54c7d3550f
|
Gezer renamed to Geziyor
|
2019-06-08 17:14:10 +03:00 |
|