2014-08-06

error.

Originally Posted By: suni123
hi

stop crawl is working
Yellow is the color of the Fetchers
This message is displayed on the Computer
'''Originally Posted By: suni123''' hi<br><br>stop crawl is working<br>Yellow is the color of the Fetchers <br> This message is displayed on the Computer

-- error
It looks like it crashed while trying to download a url. This sometimes happens. Yellow means that you had switched the fetcher
on at some point but that it is no longer running. Green means you had turned it on, and it is still running .If you had more
than one fetcher running, Yioop will eventually turn that fetcher back on itself. Since you had only one running, you need to manually
turn it back on by clicking the On link.
It looks like it crashed while trying to download a url. This sometimes happens. Yellow means that you had switched the fetcher<br>on at some point but that it is no longer running. Green means you had turned it on, and it is still running .If you had more <br>than one fetcher running, Yioop will eventually turn that fetcher back on itself. Since you had only one running, you need to manually<br>turn it back on by clicking the On link.
2014-09-29

-- error
Originally Posted By: suni123
hi,
It caused so much trouble for me what to do if there is a solution to this error
Please Help me
'''Originally Posted By: suni123''' hi,<br>It caused so much trouble for me what to do if there is a solution to this error <br>Please Help me

-- error
Posting the same image again without any additional information won't help me diagnose what's wrong.
A fetcher crash is not too critical to a crawl -- just turn it back on and it should keep crawling.
Look for what the fetcher was doing before the crash in the fetcher log and let me know.
Posting the same image again without any additional information won't help me diagnose what's wrong.<br>A fetcher crash is not too critical to a crawl -- just turn it back on and it should keep crawling.<br>Look for what the fetcher was doing before the crash in the fetcher log and let me know.
2014-10-09

-- error
Originally Posted By: suni123
Hi
Log file content fetcher :

[Thu, 09 Oct 2014 18:10:15 +0330] Downloading list of
urls...
[Thu, 09 Oct 2014 18:10:15 +0330] Fetch url list to download
time 0.0156
[Thu, 09 Oct 2014 18:10:15 +0330] So not checking scheduler.
[Thu, 09 Oct 2014 18:10:15 +0330] Current to crawl try
again count:0
[Thu, 09 Oct 2014 18:10:15 +0330] Current to crawl
count:100
[Thu, 09 Oct 2014 18:10:15 +0330] MAIN LOOP CASE 4 -- WEB
SCHEDULER
[Thu, 09 Oct 2014 18:10:15 +0330] End Name Server Check
[Thu, 09 Oct 2014 18:10:15 +0330] Done curl exec
[Thu, 09 Oct 2014 18:10:15 +0330] Set curl options for
single page request
[Thu, 09 Oct 2014 18:10:15 +0330] Init curl request of a
single page
[Thu, 09 Oct 2014 18:10:15 +0330] http://localhost/ to see
if active crawl time has changed.
[Thu, 09 Oct 2014 18:10:15 +0330] Checking name server:
[Thu, 09 Oct 2014 18:10:15 +0330] Switching archive...
[Thu, 09 Oct 2014 18:10:15 +0330] New name:
C:/xampp/htdocs/cache/0-Archive1412515961
[Thu, 09 Oct 2014 18:10:15 +0330] MAIN LOOP CASE 1 -- SWITCH
CRAWL OR NO CURRENT CRAWL
[Thu, 09 Oct 2014 18:10:15 +0330] End Name Server Check
[Thu, 09 Oct 2014 18:10:15 +0330] Dumping 95 from old
fetch to try to make a clean re-start.
[Thu, 09 Oct 2014 18:10:15 +0330] Fetch on crawl 1412515961
was not halted properly.
[Thu, 09 Oct 2014 18:10:15 +0330] Done curl exec
[Thu, 09 Oct 2014 18:10:15 +0330] Set curl options for
single page request
[Thu, 09 Oct 2014 18:10:15 +0330] Init curl request of a
single page
[Thu, 09 Oct 2014 18:10:15 +0330] http://localhost/ to see
if should start crawling
[Thu, 09 Oct 2014 18:10:15 +0330] Checking name server:
[Thu, 09 Oct 2014 18:10:15 +0330] In Fetch Loop
Initialize logger..

[Thu, 09 Oct 2014 18:10:15 +0330]
[Thu, 09 Oct 2014 17:16:17 +0330] Init Get Pages 0.0156
[Thu, 09 Oct 2014 17:16:17 +0330] Downloading list of
urls...
[Thu, 09 Oct 2014 17:16:17 +0330] Fetch url list to download
time 0
[Thu, 09 Oct 2014 17:16:17 +0330] Time to check Scheduler
0.0936
[Thu, 09 Oct 2014 17:16:17 +0330]
http://!!!.!!.!!!.!!!/?c=fetch&a=schedu ... 2862377&se
ssion=7f9b69bf6b42a3ee2660af3f414d2d35&robot_instance=0-loca
lhost-1411380798&machine_uri=/&crawl_time=1412515961&check_c
rawl_time=1412862376
[Thu, 09 Oct 2014 17:16:17 +0330] Making request:
[Thu, 09 Oct 2014 17:16:17 +0330] Done curl exec
[Thu, 09 Oct 2014 17:16:17 +0330] Set curl options for
single page request
[Thu, 09 Oct 2014 17:16:17 +0330] Init curl request of a
single page
[Thu, 09 Oct 2014 17:16:17 +0330] Checking
http://!!!.!!.!!!.!!!/ for a new schedule.
[Thu, 09 Oct 2014 17:16:17 +0330] MAIN LOOP CASE 4 -- WEB
SCHEDULER
[Thu, 09 Oct 2014 17:16:17 +0330] End Name Server Check
[Thu, 09 Oct 2014 17:16:17 +0330] Done curl exec
[Thu, 09 Oct 2014 17:16:16 +0330] Set curl options for
single page request
[Thu, 09 Oct 2014 17:16:16 +0330] Init curl request of a
single page
[Thu, 09 Oct 2014 17:16:16 +0330] http://localhost/ to see
if active crawl time has changed.
[Thu, 09 Oct 2014 17:16:16 +0330] Checking name server:
[Thu, 09 Oct 2014 17:16:13 +0330] Ensure minimum loop time
by sleeping...3
[Thu, 09 Oct 2014 17:16:13 +0330] Update Server Time
1.279202
[Thu, 09 Oct 2014 17:16:13 +0330] ... Current
Memory:6698744
[Thu, 09 Oct 2014 17:16:13 +0330] Updated Queue Server, sent
approximately 1247020 bytes:
[Thu, 09 Oct 2014 17:16:13 +0330] This fetcher peak memory
usage: 27581576
[Thu, 09 Oct 2014 17:16:13 +0330] Web Server peak memory
usage: 8498248
[Thu, 09 Oct 2014 17:16:13 +0330] Queue Server's crawl time
is: 1412515961
[Thu, 09 Oct 2014 17:16:13 +0330] Queue Server info response
code: 1

[Thu, 09 Oct 2014 17:16:13 +0330] ... Data upload complete
[Thu, 09 Oct 2014 17:16:13 +0330] Messages from Fetch
Controller:
[Thu, 09 Oct 2014 17:16:13 +0330] Done curl exec
[Thu, 09 Oct 2014 17:16:13 +0330] Set curl options for
single page request
[Thu, 09 Oct 2014 17:16:13 +0330] Init curl request of a
single page
[Thu, 09 Oct 2014 17:16:13 +0330] ...sending about 1247020
bytes.
[Thu, 09 Oct 2014 17:16:13 +0330] Sending Queue Server Part
1 of 1...
[Thu, 09 Oct 2014 17:16:13 +0330] ...
[Thu, 09 Oct 2014 17:16:13 +0330] ...1218908 bytes of index
data
[Thu, 09 Oct 2014 17:16:13 +0330] ...Finish Compressing seen
URLs.
[Thu, 09 Oct 2014 17:16:13 +0330] Saving index shard ..
wrote doc map. Done save
[Thu, 09 Oct 2014 17:16:13 +0330] Saving index shard ..
packed header
[Thu, 09 Oct 2014 17:16:13 +0330] Saving index shard .. make
prefixes
[Thu, 09 Oct 2014 17:16:13 +0330] Saving index shard .. done
merge postings to string
[Thu, 09 Oct 2014 17:16:13 +0330] ..Done Merge Index Posting
Final Copy
[Thu, 09 Oct 2014 17:16:13 +0330] ..Merge Index Posting
Final Copy
[Thu, 09 Oct 2014 17:16:13 +0330] Merge index shard postings
to string to save memory.
[Thu, 09 Oct 2014 17:16:13 +0330] Saving Mini Inverted
Index...
[Thu, 09 Oct 2014 17:16:13 +0330] Build mini inverted
index time 0.748801

For me, it happens approximately every 10 minutes.
Where the problem can be
'''Originally Posted By: suni123''' Hi <br>Log file content fetcher :<br><br>[Thu, 09 Oct 2014 18:10:15 +0330] Downloading list of<br>urls...<br>[Thu, 09 Oct 2014 18:10:15 +0330] Fetch url list to download<br>time 0.0156<br>[Thu, 09 Oct 2014 18:10:15 +0330] So not checking scheduler.<br>[Thu, 09 Oct 2014 18:10:15 +0330] Current to crawl try<br>again count:0<br>[Thu, 09 Oct 2014 18:10:15 +0330] Current to crawl<br>count:100<br>[Thu, 09 Oct 2014 18:10:15 +0330] MAIN LOOP CASE 4 -- WEB<br>SCHEDULER<br>[Thu, 09 Oct 2014 18:10:15 +0330] End Name Server Check<br>[Thu, 09 Oct 2014 18:10:15 +0330] Done curl exec<br>[Thu, 09 Oct 2014 18:10:15 +0330] Set curl options for<br>single page request<br>[Thu, 09 Oct 2014 18:10:15 +0330] Init curl request of a<br>single page<br>[Thu, 09 Oct 2014 18:10:15 +0330] http://localhost/ to see<br>if active crawl time has changed.<br>[Thu, 09 Oct 2014 18:10:15 +0330] Checking name server:<br>[Thu, 09 Oct 2014 18:10:15 +0330] Switching archive...<br>[Thu, 09 Oct 2014 18:10:15 +0330] New name:<br>C:/xampp/htdocs/cache/0-Archive1412515961<br>[Thu, 09 Oct 2014 18:10:15 +0330] MAIN LOOP CASE 1 -- SWITCH<br>CRAWL OR NO CURRENT CRAWL<br>[Thu, 09 Oct 2014 18:10:15 +0330] End Name Server Check<br>[Thu, 09 Oct 2014 18:10:15 +0330] Dumping 95 from old<br>fetch to try to make a clean re-start.<br>[Thu, 09 Oct 2014 18:10:15 +0330] Fetch on crawl 1412515961<br>was not halted properly.<br>[Thu, 09 Oct 2014 18:10:15 +0330] Done curl exec<br>[Thu, 09 Oct 2014 18:10:15 +0330] Set curl options for<br>single page request<br>[Thu, 09 Oct 2014 18:10:15 +0330] Init curl request of a<br>single page<br>[Thu, 09 Oct 2014 18:10:15 +0330] http://localhost/ to see<br>if should start crawling<br>[Thu, 09 Oct 2014 18:10:15 +0330] Checking name server:<br>[Thu, 09 Oct 2014 18:10:15 +0330] In Fetch Loop<br>Initialize logger..<br><br>[Thu, 09 Oct 2014 18:10:15 +0330] <br>[Thu, 09 Oct 2014 17:16:17 +0330] Init Get Pages 0.0156<br>[Thu, 09 Oct 2014 17:16:17 +0330] Downloading list of<br>urls...<br>[Thu, 09 Oct 2014 17:16:17 +0330] Fetch url list to download<br>time 0<br>[Thu, 09 Oct 2014 17:16:17 +0330] Time to check Scheduler<br>0.0936<br>[Thu, 09 Oct 2014 17:16:17 +0330]<br>http://!!!.!!.!!!.!!!/?c=fetch&a=schedu ... 2862377&se<br>ssion=7f9b69bf6b42a3ee2660af3f414d2d35&robot_instance=0-loca<br>lhost-1411380798&machine_uri=/&crawl_time=1412515961&check_c<br>rawl_time=1412862376<br>[Thu, 09 Oct 2014 17:16:17 +0330] Making request: <br>[Thu, 09 Oct 2014 17:16:17 +0330] Done curl exec<br>[Thu, 09 Oct 2014 17:16:17 +0330] Set curl options for<br>single page request<br>[Thu, 09 Oct 2014 17:16:17 +0330] Init curl request of a<br>single page<br>[Thu, 09 Oct 2014 17:16:17 +0330] Checking <br>http://!!!.!!.!!!.!!!/ for a new schedule.<br>[Thu, 09 Oct 2014 17:16:17 +0330] MAIN LOOP CASE 4 -- WEB<br>SCHEDULER<br>[Thu, 09 Oct 2014 17:16:17 +0330] End Name Server Check<br>[Thu, 09 Oct 2014 17:16:17 +0330] Done curl exec<br>[Thu, 09 Oct 2014 17:16:16 +0330] Set curl options for<br>single page request<br>[Thu, 09 Oct 2014 17:16:16 +0330] Init curl request of a<br>single page<br>[Thu, 09 Oct 2014 17:16:16 +0330] http://localhost/ to see<br>if active crawl time has changed.<br>[Thu, 09 Oct 2014 17:16:16 +0330] Checking name server:<br>[Thu, 09 Oct 2014 17:16:13 +0330] Ensure minimum loop time<br>by sleeping...3<br>[Thu, 09 Oct 2014 17:16:13 +0330] Update Server Time<br>1.279202<br>[Thu, 09 Oct 2014 17:16:13 +0330] ... Current<br>Memory:6698744<br>[Thu, 09 Oct 2014 17:16:13 +0330] Updated Queue Server, sent<br>approximately 1247020 bytes:<br>[Thu, 09 Oct 2014 17:16:13 +0330] This fetcher peak memory<br>usage: 27581576<br>[Thu, 09 Oct 2014 17:16:13 +0330] Web Server peak memory<br>usage: 8498248<br>[Thu, 09 Oct 2014 17:16:13 +0330] Queue Server's crawl time<br>is: 1412515961<br>[Thu, 09 Oct 2014 17:16:13 +0330] Queue Server info response<br>code: 1<br><br>[Thu, 09 Oct 2014 17:16:13 +0330] ... Data upload complete<br>[Thu, 09 Oct 2014 17:16:13 +0330] Messages from Fetch<br>Controller:<br>[Thu, 09 Oct 2014 17:16:13 +0330] Done curl exec<br>[Thu, 09 Oct 2014 17:16:13 +0330] Set curl options for<br>single page request<br>[Thu, 09 Oct 2014 17:16:13 +0330] Init curl request of a<br>single page<br>[Thu, 09 Oct 2014 17:16:13 +0330] ...sending about 1247020<br>bytes.<br>[Thu, 09 Oct 2014 17:16:13 +0330] Sending Queue Server Part<br>1 of 1...<br>[Thu, 09 Oct 2014 17:16:13 +0330] ...<br>[Thu, 09 Oct 2014 17:16:13 +0330] ...1218908 bytes of index<br>data<br>[Thu, 09 Oct 2014 17:16:13 +0330] ...Finish Compressing seen<br>URLs.<br>[Thu, 09 Oct 2014 17:16:13 +0330] Saving index shard ..<br>wrote doc map. Done save<br>[Thu, 09 Oct 2014 17:16:13 +0330] Saving index shard ..<br>packed header<br>[Thu, 09 Oct 2014 17:16:13 +0330] Saving index shard .. make<br>prefixes<br>[Thu, 09 Oct 2014 17:16:13 +0330] Saving index shard .. done<br>merge postings to string<br>[Thu, 09 Oct 2014 17:16:13 +0330] ..Done Merge Index Posting<br>Final Copy<br>[Thu, 09 Oct 2014 17:16:13 +0330] ..Merge Index Posting<br>Final Copy<br>[Thu, 09 Oct 2014 17:16:13 +0330] Merge index shard postings<br>to string to save memory.<br>[Thu, 09 Oct 2014 17:16:13 +0330] Saving Mini Inverted<br>Index...<br>[Thu, 09 Oct 2014 17:16:13 +0330] Build mini inverted<br>index time 0.748801<br><br>For me, it happens approximately every 10 minutes. <br>Where the problem can be

-- error
Was that the end of the fetcher log file for that particular run?
Was that the end of the fetcher log file for that particular run?
2014-10-21

-- error
Originally Posted By: suni123
The problem was resolved error
After crawling to the extent of the problem was that the files were having trouble finding Php.
To reload the file in PHP extention
/php/ext/..-.. .dll files This problem was solved.
And other crash error did not occur.
Thanks
'''Originally Posted By: suni123''' The problem was resolved error <br>After crawling to the extent of the problem was that the files were having trouble finding Php. <br>To reload the file in PHP extention <br>/php/ext/..-.. .dll files This problem was solved.<br>And other crash error did not occur.<br>Thanks
X