Yioop - PHP Search Engine

Encoding error.

Hi Chris. Please help! Yioop v5.0.4. I scan three sites.

1) https://www.asberg.ru/
Extracted Title is OK
Extracted Description is OK
Extracted Links Array is like [https://www.asberg.ru/comp-novostei/novuyu-lineyku-trekhfaznykh-ibp-easy-ups-3s.html] => .. ÐÐ¾Ð²Ð°Ñ Ð»Ð¸Ð½ÐµÐ¹ÐºÐ° Ñ‚Ñ€ÐµÑ…Ñ„Ð°Ð·Ð½Ñ‹Ñ… Ð˜Ð‘ÐŸ Easy UPS 3S Ð¾Ñ‚ Schneider Electric
Cache is OK
2) https://moscow.rt.ru/
Extracted Title is OK
Extracted Description is like .. Ð¦Ð¸Ñ„Ñ€Ð¾Ð²Ñ‹Ðµ ÑƒÑÐ»ÑƒÐ³Ð¸ Ð¸ ÑÐµÑ€Ð²Ð¸ÑÑ‹ Ð´Ð»Ñ Ð²Ð°ÑˆÐµÐ¹ ÑÐµÐ¼ÑŒÐ¸ ÐŸÐ¾Ð´Ñ€Ð¾Ð±Ð½ÐµÐµ Ð‘ÑƒÐ´ÑŒÑ‚Ðµ ÑÐ¿Ð¾ÐºÐ¾Ð¹Ð½Ñ‹ Ð·Ð°
Extracted Links Array is OK
Cache is OK
3) http://www.housecontrol.ru/material.php
This site has the uppercase meta-tag:
I have changed two lines in the file yioop-v5.0.4/src/library/LocaleFunctions.php in the function guessEncodingHtml: $reg = "/charset(\s*)=(\s*)(\'|\")?((\w|\-)+)(\'|\")?/iu"; mb_ereg_search($reg, "i");
"i" - this means no case sensitivity.
Now scanning site №3 works without errors. But I need your help with site №1 and site №2.

Irina.

(Edited: 2018-12-19)

Hi Chris. Please help! Yioop v5.0.4. I scan three sites. * '''1) https://www.asberg.ru/''' * Extracted Title is OK * Extracted Description is OK * Extracted Links Array is like <nowiki>[https://www.asberg.ru/comp-novostei/novuyu-lineyku-trekhfaznykh-ibp-easy-ups-3s.html] => .. ÐÐ¾Ð²Ð°Ñ Ð»Ð¸Ð½ÐµÐ¹ÐºÐ° Ñ‚Ñ€ÐµÑ…Ñ„Ð°Ð·Ð½Ñ‹Ñ… Ð˜Ð‘ÐŸ Easy UPS 3S Ð¾Ñ‚ Schneider Electric</nowiki> * Cache is OK * '''2) https://moscow.rt.ru/''' * Extracted Title is OK * Extracted Description is like <nowiki>.. Ð¦Ð¸Ñ„Ñ€Ð¾Ð²Ñ‹Ðµ ÑƒÑÐ»ÑƒÐ³Ð¸ Ð¸ ÑÐµÑ€Ð²Ð¸ÑÑ‹ Ð´Ð»Ñ Ð²Ð°ÑˆÐµÐ¹ ÑÐµÐ¼ÑŒÐ¸ ÐŸÐ¾Ð´Ñ€Ð¾Ð±Ð½ÐµÐµ Ð‘ÑƒÐ´ÑŒÑ‚Ðµ ÑÐ¿Ð¾ÐºÐ¾Ð¹Ð½Ñ‹ Ð·Ð°</nowiki> * Extracted Links Array is OK * Cache is OK * '''3) http://www.housecontrol.ru/material.php''' * This site has the uppercase meta-tag: <nowiki><META http-equiv=content-type content="TEXT/HTML; CHARSET=Windows-1251"></nowiki> * I have changed two lines in the file yioop-v5.0.4/src/library/LocaleFunctions.php in the function guessEncodingHtml: <nowiki>$reg = "/charset(\s*)=(\s*)(\'|\")?((\w|\-)+)(\'|\")?/iu";</nowiki> <nowiki>mb_ereg_search($reg, "i");</nowiki> * "i" - this means no case sensitivity. * Now scanning site №3 works without errors. But I need your help with site №1 and site №2. Irina.

-- Encoding error

Hi Irina,

I just finished the semester here. I will try to see if I can fix the encoding issue for 1 and 2 tomorrow (California Dec 21).

Best,

Chris

Hi Irina, I just finished the semester here. I will try to see if I can fix the encoding issue for 1 and 2 tomorrow (California Dec 21). Best, Chris

-- Encoding error

OK. Thank you. I will wait for your results.

-- Encoding error

Hi Irina,

I tracked down and fixed the encoding issues you had above. I also incorporated your bug fix for 3 above into the master branch of yioop in the git repository. You can obtain the development branch of yioop by typing:

 git clone https://seekquarry.com/git/yioop.git

Alternatively, you can find different versions of yioop at:

 https://www.seekquarry.com/viewgit/?a=summary&p=yioop

If you want to try to modify version 5.0.4 rather than use the development branch, download the development branch and type:

 git diff c4376bb6 512267d0

to see the changes I needed to make to fix 1 and 2. They were mainly places where I had to unscrew up the UTF-8 encoding. Let me know if these instructions suffice, if not, I can try to write something longer.

Best,

Chris

Hi Irina, I tracked down and fixed the encoding issues you had above. I also incorporated your bug fix for 3 above into the master branch of yioop in the git repository. You can obtain the development branch of yioop by typing: git clone https://seekquarry.com/git/yioop.git Alternatively, you can find different versions of yioop at: https://www.seekquarry.com/viewgit/?a=summary&p=yioop If you want to try to modify version 5.0.4 rather than use the development branch, download the development branch and type: git diff c4376bb6 512267d0 to see the changes I needed to make to fix 1 and 2. They were mainly places where I had to unscrew up the UTF-8 encoding. Let me know if these instructions suffice, if not, I can try to write something longer. Best, Chris

-- Encoding error

Oh, thanks, Chris. I am very pleased!!!

-- Encoding error

Hi Chris. Happy New Year and Merry Christmas! I have found one problem in the development branch. The link "Resume" in the menu "Manage Crawls" does not work. Http error 500 occurs. Please help!