8 Commits

Author SHA1 Message Date
Musab Gültekin
7b23596a2d Middleware support added. HTML Parsing disable option added.
Goroutine leaks will be tested using leaktest lib.
2019-06-15 17:55:40 +03:00
Musab Gültekin
6caf1effd6 Rendered field exported to support rendered requests on Do function. Data races fixed. 2019-06-14 15:23:56 +03:00
Musab Gültekin
1a7d480b36 JS Rendered requests with Chrome support added 2019-06-13 22:08:45 +03:00
Musab Gültekin
e4e8723426 Callback are now mandatory as almost all the scrapers use it. 2019-06-11 14:24:48 +03:00
Musab Gültekin
ca2414c5c8 Request callbacks added.
Recover from all panics and continue scraping.
Only parse HTML if response is HTML.
2019-06-09 21:13:30 +03:00
Musab Gültekin
a9aaf86df3 Automatic determining response and decoding it. 2019-06-09 10:46:32 +03:00
Musab Gültekin
ca197ff06a Caching added.
JSON File export will append, not truncate.
2019-06-08 15:29:09 +03:00
Musab Gültekin
6358b87472 Use parse function to parse responses, instead of channels.
Parse response as HTML Document using goquery.
Added simple README.
2019-06-06 22:48:57 +03:00