19 Commits

Author SHA1 Message Date
Musab Gültekin
7b23596a2d Middleware support added. HTML Parsing disable option added.
Goroutine leaks will be tested using leaktest lib.
2019-06-15 17:55:40 +03:00
Musab Gültekin
b2f32b8830
Merge branch 'master' into master 2019-06-14 15:32:36 +03:00
Musab Gültekin
6caf1effd6 Rendered field exported to support rendered requests on Do function. Data races fixed. 2019-06-14 15:23:56 +03:00
Ibrahim Serdar Acikgoz
7360ffa3c9
Update README.md 2019-06-14 14:57:53 +03:00
Musab Gültekin
1a7d480b36 JS Rendered requests with Chrome support added 2019-06-13 22:08:45 +03:00
Musab Gültekin
184081d3bf README updated for more advanced usage. Updated tests. 2019-06-12 22:22:01 +03:00
Musab Gültekin
d56ea161a5 Making new requests on StartRequestsFunc is simplified by using channels 2019-06-12 21:54:57 +03:00
Musab Gültekin
2f6cb06982 Disabling charset detection implemented. 2019-06-12 11:44:31 +03:00
Musab Gültekin
3790295658 Multiple Exporters and custom Exporters support added. 2019-06-11 16:10:49 +03:00
Musab Gültekin
e4e8723426 Callback are now mandatory as almost all the scrapers use it. 2019-06-11 14:24:48 +03:00
Musab Gültekin
7abc7a370d Disabling logs support added. 2019-06-09 19:14:46 +03:00
Musab Gültekin
b973c1c064 Request delays support added 2019-06-09 14:24:53 +03:00
Musab Gültekin
2263108838 User-Agent change support added. 2019-06-09 13:43:17 +03:00
Musab Gültekin
d967555b62 Global and Domain Concurrency limit implemented. Updated README 2019-06-09 11:53:40 +03:00
Musab Gültekin
2e3bd18430 Options refactored to its own file. Timeout increased to 60 sec 2019-06-08 20:36:43 +03:00
Musab Gültekin
815ae7eec5 Do request support added. Updated docs. 2019-06-08 19:45:48 +03:00
Musab Gültekin
54c7d3550f Gezer renamed to Geziyor 2019-06-08 17:14:10 +03:00
Musab Gültekin
c525e0d7d0 Don't visit already visited URLs. Update README 2019-06-08 17:04:00 +03:00
Musab Gültekin
6358b87472 Use parse function to parse responses, instead of channels.
Parse response as HTML Document using goquery.
Added simple README.
2019-06-06 22:48:57 +03:00