2014-10-06

Remove old messages.. .

Originally Posted By: mburukenya

Hi,
I think my queue server is unable to delete orphaned dirs. Is there any way I can delete them manually.

LOG FILE
Initialize logger..
[Mon, 06 Oct 2014 01:20:28 -0700] Remove old messages..
[Mon, 06 Oct 2014 01:20:28 -0700] In queue loop!! IndexerAndScheduler
[Mon, 06 Oct 2014 01:20:28 -0700] ..Indexer still deleting 1 of 3 orphaned dirs Current memory usage:4713296
[Mon, 06 Oct 2014 01:20:28 -0700] IndexerAndScheduler peak memory usage so far: 4762704!!
[Mon, 06 Oct 2014 01:20:28 -0700] IndexerAndScheduler is waiting for start message

[Mon, 06 Oct 2014 01:20:33 -0700] IndexerAndScheduler peak memory usage so far: 4762704!!
[Mon, 06 Oct 2014 01:20:33 -0700] IndexerAndScheduler is waiting for start message

[Mon, 06 Oct 2014 01:20:38 -0700] IndexerAndScheduler peak memory usage so far: 4762704!!
[Mon, 06 Oct 2014 01:20:38 -0700] IndexerAndScheduler is waiting for start message

'''Originally Posted By: mburukenya''' Hi, I think my queue server is unable to delete orphaned dirs. Is there any way I can delete them manually. LOG FILE Initialize logger.. [Mon, 06 Oct 2014 01:20:28 -0700] Remove old messages.. [Mon, 06 Oct 2014 01:20:28 -0700] In queue loop!! IndexerAndScheduler [Mon, 06 Oct 2014 01:20:28 -0700] ..Indexer still deleting 1 of 3 orphaned dirs Current memory usage:4713296 [Mon, 06 Oct 2014 01:20:28 -0700] IndexerAndScheduler peak memory usage so far: 4762704!! [Mon, 06 Oct 2014 01:20:28 -0700] IndexerAndScheduler is waiting for start message [Mon, 06 Oct 2014 01:20:33 -0700] IndexerAndScheduler peak memory usage so far: 4762704!! [Mon, 06 Oct 2014 01:20:33 -0700] IndexerAndScheduler is waiting for start message [Mon, 06 Oct 2014 01:20:38 -0700] IndexerAndScheduler peak memory usage so far: 4762704!! [Mon, 06 Oct 2014 01:20:38 -0700] IndexerAndScheduler is waiting for start message

-- Remove old messages..

That log message is actually not accurate. It should say that it is checking directory 1 of 3
to see whether any are orphaned. At the point that it writes the message, it hasn't determined
in that directory whether or not there are any orphaned bundles. So you can safely ignore that
message. I will modify what it says in the next release. Two places you could manually
look for not in use directories are:
WORK_DIRECTORY/cache
and
WORK_DIRECTORY/schedules
Folders with timestamps in their names that don't correspond to any crawl you want to keep
can be safely deleted.

That log message is actually not accurate. It should say that it is checking directory 1 of 3 to see whether any are orphaned. At the point that it writes the message, it hasn't determined in that directory whether or not there are any orphaned bundles. So you can safely ignore that message. I will modify what it says in the next release. Two places you could manually look for not in use directories are: WORK_DIRECTORY/cache and WORK_DIRECTORY/schedules Folders with timestamps in their names that don't correspond to any crawl you want to keep can be safely deleted.

Query Statistics

https://yioop.com/thread/2922

Total Elapsed Time for Queries: 0.06142044067382812 seconds.

SELECT LOCALE_NAME, WRITING_MODE FROM LOCALE WHERE LOCALE_TAG ='en-US'

Time: 0.0002000331878662109 seconds.

SELECT COALESCE(MAX(UPDATE_TIMESTAMP), 0) AS MOST_RECENT FROM ITEM_IMPRESSION_SUMMARY WHERE USER_ID = 2 AND ITEM_TYPE = 3 AND ITEM_ID IN (SELECT GROUP_ID FROM USER_GROUP WHERE USER_ID = 2 AND STATUS = 1) AND UPDATE_PERIOD = -4

Time: 0.0006909370422363281 seconds.

DELETE FROM ITEM_IMPRESSION_SUMMARY WHERE USER_ID=? AND ITEM_ID=? AND ITEM_TYPE=? AND UPDATE_PERIOD = -4
Array ( [0] => 2 [1] => 2922 [2] => 1 )

Time: 0.006061077117919922 seconds.

INSERT INTO ITEM_IMPRESSION_SUMMARY VALUES (?, ?, ?, -4, ?, 0, -1, -1) ON CONFLICT DO NOTHING
Array ( [0] => 2 [1] => 2922 [2] => 1 [3] => 1789979521 )

Time: 0.00536799430847168 seconds.

UPDATE ITEM_IMPRESSION_SUMMARY SET NUM_VIEWS = NUM_VIEWS + 1 WHERE USER_ID=? AND ITEM_ID=? AND ITEM_TYPE=? AND UPDATE_PERIOD = -2 AND UPDATE_TIMESTAMP = 0
Array ( [0] => 2 [1] => 2922 [2] => 1 )

Time: 0.0004899501800537109 seconds.

DELETE FROM ITEM_IMPRESSION_SUMMARY WHERE USER_ID=? AND ITEM_ID=? AND ITEM_TYPE=? AND UPDATE_PERIOD = -4
Array ( [0] => 2 [1] => 212 [2] => 3 )

Time: 0.004828929901123047 seconds.

INSERT INTO ITEM_IMPRESSION_SUMMARY VALUES (?, ?, ?, -4, ?, 0, -1, -1) ON CONFLICT DO NOTHING
Array ( [0] => 2 [1] => 212 [2] => 3 [3] => 1789979521 )

Time: 0.005400896072387695 seconds.

UPDATE ITEM_IMPRESSION_SUMMARY SET NUM_VIEWS = NUM_VIEWS + 1 WHERE USER_ID=? AND ITEM_ID=? AND ITEM_TYPE=? AND UPDATE_PERIOD = -2 AND UPDATE_TIMESTAMP = 0
Array ( [0] => 2 [1] => 212 [2] => 3 )

Time: 0.0005021095275878906 seconds.

SELECT COUNT(GI.ID) AS NUM FROM GROUP_ITEM GI WHERE GI.GROUP_ID IN (SELECT GROUP_ID FROM USER_GROUP WHERE USER_ID = ? AND STATUS = 1) AND GI.TITLE NOT LIKE ? AND GI.PUBDATE > ?
Array ( [0] => 2 [1] => %2% [2] => 1789979521 )

Time: 0.0004549026489257812 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-4 )

Time: 0.0002131462097167969 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-37 )

Time: 8.606910705566406E-5 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-202 )

Time: 6.794929504394531E-5 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-687 )

Time: 6.389617919921875E-5 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-1115 )

Time: 6.389617919921875E-5 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-1138 )

Time: 0.0001010894775390625 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-1140 )

Time: 6.4849853515625E-5 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-1149 )

Time: 7.510185241699219E-5 seconds.

SELECT PARENT_ID FROM GROUP_ITEM WHERE GROUP_ID=? AND USER_ID=? AND TITLE=? LIMIT 1
Array ( [0] => -1 [1] => 2 [2] => 2-1152 )

Time: 6.4849853515625E-5 seconds.

SELECT * FROM GROUP_ITEM WHERE ID=? LIMIT 1
Array ( [0] => 2922 )

Time: 0.0001809597015380859 seconds.

SELECT OPTIONS FROM SOCIAL_GROUPS WHERE GROUP_ID = ?
Array ( [0] => 212 )

Time: 8.916854858398438E-5 seconds.

SELECT COUNT(DISTINCT GI.ID) AS NUM FROM GROUP_ITEM GI, SOCIAL_GROUPS G, USER_GROUP UG, USERS O WHERE GI.PARENT_ID='2922' AND NOT LOWER(group_name) LIKE LOWER('Personal$%') AND (UG.USER_ID='2' OR G.REGISTER_TYPE IN ('4','3') ) AND GI.USER_ID=O.USER_ID AND GI.GROUP_ID=G.GROUP_ID AND GI.GROUP_ID=UG.GROUP_ID AND (( G.MEMBER_ACCESS IN ('2','3','4', '5')) OR (G.OWNER_ID = UG.USER_ID OR UG.USER_ID = '1'))

Time: 0.002898216247558594 seconds.

SELECT DISTINCT GI.ID AS ID, GI.PARENT_ID AS PARENT_ID, GI.GROUP_ID AS GROUP_ID, GI.TITLE AS TITLE, GI.DESCRIPTION AS DESCRIPTION, GI.FLAG AS FLAG, GI.PUBDATE AS PUBDATE, GI.EDIT_DATE AS EDIT_DATE, G.OWNER_ID AS OWNER_ID, G.MEMBER_ACCESS AS MEMBER_ACCESS, G.GROUP_NAME AS GROUP_NAME, P.USER_NAME AS USER_NAME, P.USER_ID AS USER_ID, GI.TYPE AS TYPE, GI.UPS AS UPS, GI.DOWNS AS DOWNS, G.VOTE_ACCESS AS VOTE_ACCESS FROM GROUP_ITEM GI, SOCIAL_GROUPS G, USER_GROUP UG, USERS P WHERE GI.PARENT_ID='2922' AND NOT LOWER(group_name) LIKE LOWER('Personal$%') AND (UG.USER_ID='2' OR G.REGISTER_TYPE IN ('4','3') ) AND GI.GROUP_ID=G.GROUP_ID AND GI.GROUP_ID=UG.GROUP_ID AND (( G.MEMBER_ACCESS IN ('2','3','4', '5')) OR (G.OWNER_ID = UG.USER_ID OR UG.USER_ID = '1')) AND GI.PARENT_ID NOT IN (SELECT DISCUSS_THREAD FROM GROUP_PAGE WHERE TITLE LIKE '%$$%' AND DISCUSS_THREAD IS NOT NULL) AND P.USER_ID = GI.USER_ID ORDER BY GI.PUBDATE ASC LIMIT 10 OFFSET 0

Time: 0.03038907051086426 seconds.

SELECT OPTIONS FROM SOCIAL_GROUPS WHERE GROUP_ID = ?
Array ( [0] => 212 )

Time: 0.00018310546875 seconds.

SELECT OPTIONS FROM SOCIAL_GROUPS WHERE GROUP_ID = ?
Array ( [0] => 212 )

Time: 2.908706665039062E-5 seconds.

SELECT * FROM GROUP_ITEM WHERE ID=? LIMIT 1
Array ( [0] => 2922 )

Time: 0.0001990795135498047 seconds.

SELECT OPTIONS FROM SOCIAL_GROUPS WHERE GROUP_ID = ?
Array ( [0] => 212 )

Time: 4.792213439941406E-5 seconds.

SELECT STATUS FROM USERS WHERE USER_ID = ?
Array ( [0] => 2 )

Time: 0.0002171993255615234 seconds.

SELECT USER_NAME FROM USERS WHERE USER_ID = ?
Array ( [0] => 2 )

Time: 0.0001521110534667969 seconds.

SELECT * FROM VISITOR WHERE ADDRESS = :address AND PAGE_NAME = :page_name LIMIT 1
Array ( [:address] => 216.73.216.124 [:page_name] => forbidden_time_out )

Time: 0.0002870559692382812 seconds.

SELECT COUNT(DISTINCT G.GROUP_ID) AS NUM FROM USER_GROUP UG, SOCIAL_GROUPS G WHERE UG.USER_ID = ? AND UG.GROUP_ID = G.GROUP_ID AND ( UG.STATUS = 1 OR UG.STATUS = 5)
Array ( [0] => 2 )

Time: 0.0005259513854980469 seconds.

SELECT G.GROUP_ID AS GROUP_ID FROM SOCIAL_GROUPS G WHERE G.GROUP_NAME = ?
Array ( [0] => Personal$2 )

Time: 0.0002689361572265625 seconds.

SELECT USER_ID FROM USER_GROUP WHERE GROUP_ID = ?
Array ( [0] => -1 )

Time: 0.000125885009765625 seconds.

SELECT G.GROUP_ID AS GROUP_ID, G.GROUP_NAME AS GROUP_NAME, G.OWNER_ID AS OWNER_ID, O.USER_NAME AS OWNER, REGISTER_TYPE, UG.STATUS AS STATUS, G.MEMBER_ACCESS AS MEMBER_ACCESS, G.VOTE_ACCESS AS VOTE_ACCESS, G.POST_LIFETIME AS POST_LIFETIME, UG.JOIN_DATE AS JOIN_DATE, G.OPTIONS AS OPTIONS, G.RENDER_ENGINE AS RENDER_ENGINE, G.GROUP_THEME AS GROUP_THEME, G.PAGE_HEADER AS PAGE_HEADER, G.PAGE_FOOTER AS PAGE_FOOTER FROM SOCIAL_GROUPS G, USERS O, USER_GROUP UG WHERE (UG.USER_ID = :user_id) AND UG.GROUP_ID= :group_id AND UG.GROUP_ID=G.GROUP_ID AND OWNER_ID = O.USER_ID LIMIT 1
Array ( [:group_id] => 212 [:user_id] => 2 )

Time: 0.0003871917724609375 seconds.

SELECT RENDER_ENGINE FROM SOCIAL_GROUPS WHERE GROUP_ID = ?
Array ( [0] => -2 )

Time: 0.000202178955078125 seconds.

SELECT RENDER_ENGINE FROM SOCIAL_GROUPS WHERE GROUP_ID = ?
Array ( [0] => 212 )

Time: 3.004074096679688E-5 seconds.

SELECT RENDER_ENGINE FROM SOCIAL_GROUPS WHERE GROUP_ID = ?
Array ( [0] => 212 )

Time: 7.486343383789062E-5 seconds.

SELECT G.GROUP_ID AS GROUP_ID, G.GROUP_NAME AS GROUP_NAME, G.OWNER_ID AS OWNER_ID, O.USER_NAME AS OWNER, REGISTER_TYPE, UG.STATUS AS STATUS, G.MEMBER_ACCESS AS MEMBER_ACCESS, G.VOTE_ACCESS AS VOTE_ACCESS, G.POST_LIFETIME AS POST_LIFETIME, UG.JOIN_DATE AS JOIN_DATE, G.OPTIONS AS OPTIONS, G.RENDER_ENGINE AS RENDER_ENGINE, G.GROUP_THEME AS GROUP_THEME, G.PAGE_HEADER AS PAGE_HEADER, G.PAGE_FOOTER AS PAGE_FOOTER FROM SOCIAL_GROUPS G, USERS O, USER_GROUP UG WHERE (UG.USER_ID = :user_id OR G.REGISTER_TYPE IN (3,4)) AND UG.GROUP_ID= :group_id AND UG.GROUP_ID=G.GROUP_ID AND OWNER_ID = O.USER_ID LIMIT 1
Array ( [:group_id] => 212 [:user_id] => 2 )

Time: 0.0002658367156982422 seconds.

SELECT STATUS FROM USER_GROUP WHERE USER_ID=? AND GROUP_ID=? LIMIT 1
Array ( [0] => 2 [1] => 212 )

Time: 6.890296936035156E-5 seconds.