GoSƐ logo

GoSƐ

  •  0 ratings
In category: File Transfer / Single-click & Drag-n-drop Upload

About GoSƐ

GoSƐ is a modern file-uploader focusing on scalability and simplicity. It only depends on a S3 storage backend and hence scales horizontally without the need for additional databases or caches.

  •   457  
  •   0  
  •   0  
  •   0  
Github stats:
  •  Commits: N/A  
  •   N/A  
  •   N/A  
  •  Latest commit: N/A  

Deploy this app to Linode with a free $100 credit!

Languages/Platforms/Technologies:
Lincenses:

More about GoSƐ

GoSƐ logo

GoSƐ - A terascale file-uploader

goreportcard Codacy grade License GitHub go.mod Go version Go Reference

GoSƐ is a modern and scalable file-uploader focusing on scalability and simplicity. It is a little hobby project I’ve been working on over the last weekends.

The only requirement for GoSƐ is a S3 storage backend which allows to it to scale horizontally without the need for additional databases or caches. Uploaded files a divided into equally sized chunks which are hashed with a MD5 digest in the browser for upload. This allows GoSƐ to skip chunks which already exist. Seamless resumption of interrupted uploads and storage savings are the consequence.

And either way both upload and downloads are always directed directly at the S3 server so GoSƐ only sees a few small HTTP requests instead of the bulk of the data. Behind the scenes, GoSƐ uses many of the more advanced S3 features like Multi-part Uploads and Pre-signed Requests to make this happen.

Users have a few options to select between multiple pre-configured S3 buckets/servers or enable browser & mail notifications about completed uploads. A customisable retention / expiration time for each upload is also selectable by the user and implemented by S3 life-cycle policies. Optionally, users can also opt-in to use an external service to shorten the URL of the uploaded file.

Currently a single concurrent upload of a single file is supported. Users can observe the progress via a table of details statistics, a progress-bar and a chart showing the current transfer speed.

GoSƐ aims at keeping its deployment simple and by bundling both front- & backend components in a single binary or Docker image. GoSƐ has been tested with AWS S3, Ceph’s RadosGW and Minio. Pre-built binaries and Docker images of GoSƐ are available for all major operating systems and architectures at the release page.

GoSƐ is open-source software licensed under the Apache 2.0 license.

Check our my blog article for more background info.

Features

  • De-duplication of uploaded files based on their content-hash
    • Uploads of existing files will complete in no-time without re-upload
  • S3 Multi-part uploads
    • Resumption of interrupted uploads
  • Drag & Drop of files
  • Browser notifications about failed & completed uploads
  • User-provided object expiration/retention time
  • Copy URL of uploaded file to clip-board
  • Detailed transfer statistics and progress-bar / chart
  • Installation via single binary or container
    • JS/HTML/CSS Frontend is bundled into binary
  • Scalable to multiple replicas
    • All state is kept in the S3 storage backend
    • No other database or cache is required
  • Direct up & download to Amazon S3 via presigned URLs
    • Gose deployment does not see an significant traffic
  • UTF-8 filenames
  • Multiple user-selectable buckets / servers
  • Optional link shortening via an external service
  • Optional notification about new uploads via shoutrrr
    • Mail notifications to user-provided recipient
  • Cross-platform support:
    • Operating systems: Windows, macOS, Linux, BSD
    • Architectures: arm64, amd64, armv7, i386

Roadmap

Checkout the Github issue tracker.

Demo (click for Live-Demo)

Gose demo screencast

Installation

Pre-compiled binaries from GitHub releases

Take the download link for your OS/Arch from the Releases Page and run:

sudo wget https://github.com/stv0g/gose/releases/download/v0.0.2/gose_0.0.2_linux_amd64 -O /usr/local/bin/gose
chmod +x /usr/local/bin/gose

Kubernetes / Kustomize

  1. Copy default configuration file: cp config.yaml kustomize/config.yaml
  2. Adjust config: nano kustomize/config.yaml
  3. Apply configuration: kubectl apply -k kustomize

Docker

Via environment variables in .env file:

docker run --env-file=.env --publish=8080:8080 ghcr.io/stv0g/gose

or via a configuration file:

docker run -v$(pwd)/config.yaml:/config.yaml --publish=8080:8080 ghcr.io/stv0g/gose -config /config.yaml

Docker Compose

We ship a docker-compose.yml file to get you started. Please adjust the environment variables in it and then run:

docker-compose up -d

Configuration

Gose can be configured via a configuration file and/or environment variables

File

For reference have a look at the example configuration file.

Environment variables

All settings from the configuration file can also be set via environment variables:

Variable Default Value Description
GOSE_LISTEN ":8080" Listen address and port of Gose
GOSE_BASE_URL "http://localhost:8080" Base URL at which Gose is accessible
GOSE_STATIC "./dist" Directory of frontend assets (pre-compiled binaries of GoSƐ come with assets embedded into binary.)
GOSE_BUCKET gose-uploads Name of S3 bucket
GOSE_ENDPOINT (without http(s):// prefix, but with port number) Hostname:Port of S3 server
GOSE_REGION us-east-1 Region of S3 server
GOSE_PATH_STYLE false Prepend bucket name to path
GOSE_NO_SSL false Disable SSL encryption for S3
GOSE_ACCESS_KEY S3 Access Key
GOSE_SECRET_KEY S3 Secret Key
GOSE_SETUP_BUCKET true Create S3 bucket if do not exists
GOSE_SETUP_CORS true (if supported by S3 implementation) Setup S3 bucket CORS rules
GOSE_SETUP_LIFECYCLE true (if supported by S3 implementation) Setup S3 bucket lifecycle rules
GOSE_SETUP_ABORT_INCOMPLETE_UPLOADS 31 Number of days after which incomplete uploads are cleaned-up (set to 0 to disable)
GOSE_MAX_UPLOAD_SIZE 1TB Maximum upload size
GOSE_PART_SIZE 16MB Part-size for multi-part uploads
AWS_ACCESS_KEY_ID alias for GOSE_ACCESS_KEY
AWS_SECRET_ACCESS_KEY alias for GOSE_SECRET_KEY

Configuration of link shortener and notifiers must be done via a configuration file.

Author

GoSƐ has been written by Steffen Vogel.

License

GoSƐ is licensed under the Apache 2.0 license.

Comments (0)

Please login to join the discussion on this project.

GoSƐ Reviews (0)

Overall Rating

None

based on 0 ratings

Please login to review this project.

No reviews for this project yet.

↑ back to top

Linux VPS from $11/yr.
RackNerd VPS for $11.38/mo

Popular Projects

FluxBB

in Social Networks and Forums
 31k    0    0    0  

Nextcloud

in File Transfer & Synchronization
 19k    1    1    0  

Libreddit

in Social Networks and Forums
 6k    0    1    0  

CasaOS

in Self-hosting Solutions
 5k    0    0    0  

Audiobookshelf

in Audio Streaming
 4k    0    1    0  

Mediagoblin

in Photo and Video Galleries
 4k    0    0    0  

Dashboard

in Personal Dashboards
 3k    0    0    0  

Most Discussed

Nextcloud

in File Transfer & Synchronization
 19k    1    1    0  

Tube Archivist

in Automation
 3k    0    1    0  

OneDev

in Project Management
 2k    0    0    0  

iodine

in Proxy
 2k    0    0    0  

Alf.io

in Booking and Scheduling
 2k    0    0    0  

sysPass

in Password Managers
 896    0    0    0  

Misskey

in Social Networks and Forums
 2k    0    0    0  
pCloud Lifetime

Top Rated Projects

Gitea

 1 rating
in Project Management

Bagisto

 1 rating
in E-commerce

LinkAce

 1 rating
in Bookmarks and Link Sharing

Pydio

 1 rating
in File Transfer & Synchronization

Audiobookshelf

 1 rating
in Audio Streaming

Nextcloud

 1 rating
in File Transfer & Synchronization

Seafile

 1 rating
in File Transfer & Synchronization

Categories

You May Also Be Interested In

ProjectSend logo
ProjectSend cover

ProjectSend

Upload files and assign them to specific clients you create…

XBackBone logo
XBackBone cover

XBackBone

A simple, fast and lightweight file manager with instant sh…

Void logo
Void cover

Void

Lightweight, fast and elegant file hosting service for Shar…