4 Commits

Author	SHA1	Message	Date
Musab Gültekin	cfb16fe1ee	Call ErrorFunc on errors. Unexport DoRequestClient and DoRequestChrome	2019-12-13 00:03:44 +03:00
Musab Gültekin	d19465c44a	Robotstxt metrics added.	2019-07-08 14:51:54 +03:00
Musab Gültekin	d3c4389c46	Retrying support added for chrome. Fixed robots.txt retry issue. Fixed Meta issue	2019-07-07 19:50:15 +03:00
Musab Gültekin	42faa92ece	Robots.txt support implemented	2019-07-06 16:18:03 +03:00