gitea/geziyor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Request callbacks added.

Browse Source 
Recover from all panics and continue scraping.
Only parse HTML if response is HTML.
This commit is contained in:
Musab Gültekin
2019-06-09 21:13:30 +03:00
parent 7abc7a370d
commit ca2414c5c8
5 changed files with 48 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
go.mod
View File

@ -4,6 +4,8 @@ go 1.12
require (
github.com/PuerkitoBio/goquery v1.5.0
github.com/antchfx/htmlquery v1.0.0 // indirect
github.com/antchfx/xpath v1.0.0 // indirect
github.com/fpfeng/httpcache v0.0.0-20181220155740-6b8f16a92be3
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a
golang.org/x/text v0.3.2 // indirect