Musab Gültekin
ddff3aee25
Request cancellations support added to Middlewares.
...
Some core functions refactored as middlewares.
Fixed race condition in exporting system. Now, only one goroutine will be responsible for exporting. This fixes concurrency issues on writing.
2019-06-15 22:27:46 +03:00
Musab Gültekin
7b23596a2d
Middleware support added. HTML Parsing disable option added.
...
Goroutine leaks will be tested using leaktest lib.
2019-06-15 17:55:40 +03:00
Musab Gültekin
f5b3b0d049
Fixed race conditions on exporters.
...
MaxIdleConns limit disabled to support unlimited requests to all hosts.
MaxIdleConnsPerHost limit increased to speed up requests to same host.
2019-06-14 16:10:36 +03:00
Musab Gültekin
6caf1effd6
Rendered field exported to support rendered requests on Do function. Data races fixed.
2019-06-14 15:23:56 +03:00
Musab Gültekin
76a687e193
Do function refactored
2019-06-13 20:26:07 +03:00
Musab Gültekin
2f6cb06982
Disabling charset detection implemented.
2019-06-12 11:44:31 +03:00
Musab Gültekin
a311a0f998
CSV exporter support added. Not finished for map type.
2019-06-11 20:42:22 +03:00
Musab Gültekin
bbdc3bcacd
Exporters made optional, as some scrapers only want to see data in console.
2019-06-11 18:59:37 +03:00
Musab Gültekin
3790295658
Multiple Exporters and custom Exporters support added.
2019-06-11 16:10:49 +03:00
