|
|
d9ac07754f
|
more fixes
|
2022-12-23 16:00:48 +01:00 |
|
|
|
85d73be641
|
fix test/code validation issues
|
2022-12-23 15:51:54 +01:00 |
|
|
|
738852f932
|
Add custom actions for rendered requests & Fix not closing bug
|
2022-04-29 03:05:31 +03:00 |
|
|
|
34d17a2d3d
|
Merge pull request #44 from harnnless/patch-1
Fix HTTP2 support fix #43
|
2021-12-11 14:04:10 +03:00 |
|
|
|
b0fd08c670
|
Fix HTTP2 support
|
2021-12-11 18:05:17 +08:00 |
|
|
|
369b42cbc6
|
Merge pull request #39 from Walker088/wrap_post_method_support
Feature: Implement Geziyor.Post which wraps the httpClient (POST)
|
2021-10-21 22:19:25 +03:00 |
|
|
|
88238010b2
|
fix: DeepSource, Unused parameter detected in function
|
2021-10-21 09:40:09 -03:00 |
|
|
|
b1e4683037
|
Feature: Implement Geziyor.Post which wraps the httpClient(POST)
1. Implement Geziyor.Post by the same style of Geziyor.Head
2. Add two examples in geziyor_test (TestPostJson, TestPostFormUrlEncoded)
issue #38
|
2021-10-21 09:06:34 -03:00 |
|
|
|
6415a775f4
|
Fix exporter bug
|
2021-10-14 21:54:46 +03:00 |
|
|
|
b8bda36f92
|
JoinURL deprecated
|
2021-10-05 22:13:00 +03:00 |
|
|
|
019fe62883
|
Merge pull request #37 from geziyor/proxy-support
Proxy Support
|
2021-10-05 21:59:10 +03:00 |
|
|
|
97ecb7f118
|
Proxy support
|
2021-09-24 16:15:20 +03:00 |
|
|
|
110394a753
|
Add .deepsource.toml
|
2021-08-30 18:29:32 +00:00 |
|
|
|
242b025c9a
|
Set cookie test
|
2021-08-08 22:08:47 +03:00 |
|
|
|
53a91d63d6
|
Merge pull request #29 from albertbronsky/fix-remote-allocator
fixed empty context in call to NewRemoteAllocator
|
2021-08-08 21:56:27 +03:00 |
|
|
|
fc67cec165
|
Update chromedp
|
2021-08-08 21:29:08 +03:00 |
|
|
|
f73f83e493
|
fixed empty context in call to NewRemoteAllocator
|
2021-08-08 14:07:08 +03:00 |
|
|
|
d3bdaf6240
|
Added documentation and tests for request.Meta
|
2021-05-30 10:43:54 +03:00 |
|
|
|
a2a91b7b2e
|
Default allocator options are used on rendered scraping. It can be changed using custom Client or changing client options after scraper creation.
|
2021-05-23 23:48:55 +03:00 |
|
|
|
5aa2c2540e
|
Default client function moved to client_test.go as its only used there.
|
2021-05-23 23:47:43 +03:00 |
|
|
|
f35d34bc02
|
chromedp library updated.
|
2021-05-23 23:14:47 +03:00 |
|
|
|
16265e524d
|
Response.JoinURL simplified.
|
2021-05-18 13:31:23 +03:00 |
|
|
|
3c9a3849e2
|
Start command now waits for synchronized requests too. This fixes if requests are made using different goroutines with synchronized requests.
It doesn't cause any issues on concurrent requests because we already wait for them.
|
2021-04-19 12:58:47 +03:00 |
|
|
|
d28beca57a
|
Fix race condition on hosts semaphore
|
2021-04-17 14:46:45 +03:00 |
|
|
|
c527d0b885
|
SIGINT (interrupt) signal receiving refactored and fixed working on some conditions
|
2021-04-17 14:11:17 +03:00 |
|
|
|
6a23efd175
|
JoinURL now returns *url.URL and error
|
2021-04-17 11:12:22 +03:00 |
|
|
|
9ea67b3554
|
Use fmt.Errorf instead of errors package. This is good convention after go 1.13
|
2021-04-17 11:11:29 +03:00 |
|
|
|
fbee722a38
|
Rate limiting per second implemented
|
2021-04-16 15:31:31 +03:00 |
|
|
|
d8252092f7
|
Add duplicate_requests_test.go
|
2021-04-16 14:43:42 +03:00 |
|
|
|
be4d13c0ef
|
Retry checking refactored using util function.
|
2021-04-14 09:32:42 +03:00 |
|
|
|
46c4db6b1a
|
Exporters now need to return error. This is done because of simple error logging.
|
2021-04-14 09:30:17 +03:00 |
|
|
|
e3d79e2574
|
Added custom logger. Right now, not configurable.
|
2021-04-13 23:36:42 +03:00 |
|
|
|
129402d754
|
Updated chromedp
|
2021-01-28 20:50:25 +03:00 |
|
|
|
9b266b6cce
|
Allocator options added
|
2021-01-28 20:49:01 +03:00 |
|
|
|
29c29235ae
|
Fixed response error if retrying disabled
|
2020-09-05 17:24:22 +03:00 |
|
|
|
7a76a9b95e
|
Allocators seperated for transparency. Updated chrome library.
|
2020-09-05 16:14:41 +03:00 |
|
|
|
cfb16fe1ee
|
Call ErrorFunc on errors. Unexport DoRequestClient and DoRequestChrome
|
2019-12-13 00:03:44 +03:00 |
|
|
|
7d2fe57bab
|
Added error logging for HTML parser.
|
2019-12-11 13:55:38 +03:00 |
|
|
|
cbca22fefb
|
Updated chrome protocol library
|
2019-11-16 20:34:57 +03:00 |
|
|
|
6645820408
|
Added logging on allowed domains middleware and duplicate requests
|
2019-11-16 20:34:09 +03:00 |
|
|
|
9b8a3837bd
|
Added response joinURL test and updated chromedp.
|
2019-09-13 14:34:29 +03:00 |
|
|
|
3264057679
|
Fixed issue on JoinURL
|
2019-08-06 17:21:41 +03:00 |
|
|
|
86d4e80596
|
Added user-agent test, Fixed failing test
|
2019-08-05 16:18:44 +03:00 |
|
|
|
85597219e6
|
Refactored client options
Fixed default User-Agent string not being set.
|
2019-08-05 15:42:30 +03:00 |
|
|
|
0e5230eac8
|
Remote endpoint support added for js rendered requests. Geziyor is beta now.
|
2019-08-05 15:14:47 +03:00 |
|
|
|
c117d71fef
|
Updated license
|
2019-08-05 15:01:48 +03:00 |
|
|
|
32077d8433
|
Updated docs for rendered requests
|
2019-07-26 16:40:42 +03:00 |
|
|
|
e07ef4d66d
|
Fixed important bug on rendering that was causing client request made too. Updated chromedp dependency
|
2019-07-26 16:07:09 +03:00 |
|
|
|
762854e511
|
Go 1.10 and 1.11 support added by using different methods on reflect package.
|
2019-07-21 12:08:41 +03:00 |
|
|
|
df37629d4d
|
Disabled indenting on JSON exporter as it looks so ugly on exported data.
JSONLine still supports indenting.
|
2019-07-14 03:37:52 +03:00 |
|
