librengine logo

librengine

  •  0 ratings
In category: Search Engines

About librengine

Private web search engine.

  •   392  
  •   0  
  •   0  
  •   0  
Github stats:
  •  Commits: N/A  
  •   N/A  
  •   N/A  
  •  Latest commit: N/A  

Deploy this app to Linode with a free $100 credit!

Languages/Platforms/Technologies:
Lincenses:

More about librengine

Privacy Web Search Engine

Website

Features

Crawler

  • Multithreading
  • Cache
  • Robots.txt
  • Proxy
  • Queue (BFS)
  • Detect Trackers
  • Http -> Https

Website / CLI

  • Encryption (rsa)
  • API
  • Proxy
  • Nodes
  • Rating

Usage (Docker)

Please run the build every time to change the arguments.
The site is launched by default on port 8080 AND with tor proxy (!!!), to edit it you need to change config.json and rebuild website.
The api key for the database must be changed in the config and when the database is started(--api-key).

DB - please run before using other

sudo docker pull typesense/typesense:0.24.0.rcn6
mkdir /tmp/typesense-data
sudo docker run -p 8108:8108 -v/tmp/data:/data typesense/typesense:0.24.0.rcn6 --data-dir /data --api-key=xyz

Crawler

sudo docker-compose build crawler --build-arg SITES="$(cat sites.txt)"  --build-arg THREADS=1 --build-arg CONFIG="$(cat config.json)"
sudo docker-compose up crawler

Website

sudo docker-compose build website --build-arg CONFIG="$(cat config.json)"
sudo docker-compose up website

Usage (Manual)

Deps

cd scripts && sh install_deps.sh

Build

cd scripts && sh build_all.sh

Run

The site is launched by default on port 8080 AND with tor proxy (!!!), to edit it you need to change config.json.
The api key for the database must be changed in the config and when the database is started(--api-key).

DB - please run before using other

mkdir /tmp/typesense-data &&
./typesense-server --data-dir=/tmp/typesense-data --api-key=xyz --enable-cors &&
sh scripts/init_db.sh

Crawler

./crawler ../../sites.txt 5 ../../config.json
#[sites_path] [threads_count] [config path]

Website

./website ../../config.json
#[config path]

Instances

¯\(ツ)

TODO

  • [x] Docker
  • [x] Encryption (assymetric)
  • [x] Multithreading crawler
  • [ ] Robots Rules (from headers & html) & crawl-delay
  • [x] Responsive web design
  • [ ] Own FTS (...)
  • [ ] Images Crawler

Dependencies

Config

./config.json

Mirrors

License

GNU Affero General Public License v3.0

Comments (0)

Please login to join the discussion on this project.

librengine Reviews (0)

Overall Rating

None

based on 0 ratings

Please login to review this project.

No reviews for this project yet.

↑ back to top

pCloud Lifetime

Popular Projects

FluxBB

in Social Networks and Forums
 33k    0    0    0  

Nextcloud

in File Transfer & Synchronization
 20k    1    1    0  

Libreddit

in Social Networks and Forums
 7k    0    1    0  

Dashboard

in Personal Dashboards
 6k    0    0    0  

Audiobookshelf

in Audio Streaming
 5k    0    1    0  

CasaOS

in Self-hosting Solutions
 5k    0    0    0  

Mediagoblin

in Photo and Video Galleries
 4k    0    0    0  

Most Discussed

Nextcloud

in File Transfer & Synchronization
 20k    1    1    0  

Tube Archivist

in Automation
 3k    0    1    0  

OneDev

in Project Management
 2k    0    0    0  

iodine

in Proxy
 2k    0    0    0  

Alf.io

in Booking and Scheduling
 2k    0    0    0  

sysPass

in Password Managers
 1k    0    0    0  

Misskey

in Social Networks and Forums
 2k    0    0    0  
Linux VPS from $11/yr.
RackNerd VPS for $11.38/mo

Top Rated Projects

Gitea

 1 rating
in Project Management

Bagisto

 1 rating
in E-commerce

LinkAce

 1 rating
in Bookmarks and Link Sharing

Pydio

 1 rating
in File Transfer & Synchronization

Audiobookshelf

 1 rating
in Audio Streaming

Nextcloud

 1 rating
in File Transfer & Synchronization

Seafile

 1 rating
in File Transfer & Synchronization

Categories

You May Also Be Interested In

ZincSearch logo
ZincSearch cover

ZincSearch

A lightweight alternative to elasticsearch that requires mi…

Jina logo
Jina cover

Jina

Cloud-native neural search framework for any kind of data.

Yacy logo
Yacy cover

Yacy

Peer based, decentralized search engine server.