Yioop - PHP Search Engine

2016-12-02

How to use multiple drives/directories properly.

I'm running yioop on a debian server running apache. Currently everything is on the root drive, which is getting full. How can I properly use a second drive (mounted in /mnt/vdb for me) without causing issues?

2017-01-10

-- How to use multiple drives/directories properly

I would split things across drives according to what needs to be fastest. The work_directory/cache folder stores saved crawls. In it you will see folders of the form:

 #-Archive##########,
 IndexData##########,
 QueueBundle##########

These store respectively, whole caches of pages, the inverted index plus summaries for snippets, and what to crawl next for a given crawl. The minimum you need to keep in order to serve search results is the IndexData folder. Within this folder, there are three subfolders: dictionary, posting_doc_shards, and summaries. This is also the order of the folders in terms of importance for speed. If you want to split these folders across drives you can use symbolic links as long as you make sure you give the web server permissions to follow links.

Hope this helps, Chris

(Edited: 2017-01-10)

I would split things across drives according to what needs to be fastest. The work_directory/cache folder stores saved crawls. In it you will see folders of the form: #-Archive##########, IndexData##########, QueueBundle########## These store respectively, whole caches of pages, the inverted index plus summaries for snippets, and what to crawl next for a given crawl. The minimum you need to keep in order to serve search results is the IndexData folder. Within this folder, there are three subfolders: dictionary, posting_doc_shards, and summaries. This is also the order of the folders in terms of importance for speed. If you want to split these folders across drives you can use symbolic links as long as you make sure you give the web server permissions to follow links. Hope this helps, Chris