Build Cloud File Download & Manage Site using FileBrowserAria2AriaNgRcloneCaddyGoogle Drive

5/29/2021

Build Cloud File Download & Manage Site using FileBrowser+Aria2+AriaNg+Rclone+Caddy+Google Drive
This post is regarding how to use wahyd4/aria2-ui's docker image to achieve a downloading and file managing / sharing center. In just one docker image, you can do downloading (Aria2), Web GUI management for your downloading task (AriaNG) , File Managing (FileBrowser), supporting multi-type file view, and saving to Google Drive.

Pre-requirement:

Install Docker and Portainer:

Post: Install Docker, Docker-Compose, Portainer & Nginx on CentOS8

Open Firewall port at cloud firewall and local firewall if enabled:

for tcp 8000

#CentOS 6

iptables -A INPUT -p tcp --dport 8000 -j ACCEPT

service iptables save

service iptables restart

#CentOS 7

firewall-cmd --zone=public --add-port=8000/tcp --permanent

firewall-cmd --reload

Github Project and Docker Image Information

Github project :

wahyd4/aria2-ariang-docker

Docker images:

wahyd4/aria2-ui:latest

Other related GitHub Sites:

Features

Aria2 (SSL support)
AriaNg
Rclone
File Browser: Files mangement and videos playing
Auto HTTPS （Let's Encrypt）
Bind non root user into container, so non root user can also manage downloaded files.
Basic Auth
Support ARM CPUs as well, all supported CPU platforms can be found here
Cloud Storage platforms synchronization

Quick Run for Testing

docker run -d --name aria2-ui -p 80:80 wahyd4/aria2-ui

Here are default URL and user name / password.

Aria2: http://yourip
FileManger: http://yourip/files
Rclone: http://yourip/rclone
Please use admin/admin as username and password to login Filebrowser for the first time. And if rclone enabled, please use user/password to login Rclone if you didn't update ARIA2_USER and ARIA2_PWD
ENABLE_RCLONE set to false to disable Rclone in the docker

Modified Docker Run Command Based Requirements

Quick run is not exactly what we want. For my configuration, I would like to put this docker into different port, 8000, also I will need to map my host folders into docker. No need rclone to run in the docker, I will get it running on my host to mount my Google Drive. Also I will need some kind of authentication to be enabled for AriaNG web gui.

docker run -d --name aria2-ui \
  --restart=unless-stopped \
  -p 8000:80 \
  -e ENABLE_AUTH=true \
  -e ARIA2_SSL=false \
  -e ARIA2_USER=user \
  -e ARIA2_PWD=password \
  -e ARIA2_EXTERNAL_PORT=8000 \
  -e ENABLE_RCLONE=False \
  -v ~/data:/data \
  -v ~/gdrive:/gdrive \
  wahyd4/aria2-ui

Notes:

/data folder is aria2 downloading folder. After finished downloading, file will be transferred to /gdrive folder automatically with a script.
/gdrive folder is host's Rclone mounted Google Drive folder， which is mapped to ~/gdrive folder (/root/gdrive).

Configure FileBrowser to Manage Gdrive folder

This is an option to do if you would like FileBrowser web gui to manage your mounted google drive. Default it is managing download folder /data.

Change File Browser's folder. vi /app/Procfile
Change default /data folder to /gdrive. So File Browser will open /gdrive when you log in. You can mange your Google Drive from File Browser directly. It is not necessary to do this step if you does not want to manage your Google Drive folder from FileBrowser page.

[root@centos-nextcloud-aria2 ~]docker exec -it aria2-ui /bin/bash
bash-5.0# cd /app
bash-5.0# ls
Procfile aria2c.sh caddy.sh conf filebrowser filebrowser.db forego init.sh start.sh
bash-5.0# vi Procfile
filebrowser: /app/filebrowser -p 8080 -d /app/filebrowser.db -r /gdrive
caddy: /app/caddy.sh
aria2c: /app/aria2c.sh [root@centos-nextcloud-aria2 ~]# docker restart aria2-ui

Configure Aria2 to Move Completed File to /Gdrive

[root@centos-nextcloud-aria2 ~]docker exec -it aria2-ui /bin/bash

Inside the docker, create a shell script , rcloneupload.sh,with following code：

bash-4.3#vi /app/conf/rcloneupload.sh

#!/bin/bash

GID="$1";
FileNum="$2";
File="$3";
MinSize="5"  #限制最低上传大小，默认5k
MaxSize="157286400"  #限制最高文件大小(单位k)，默认15G
RemoteDIR="/gdrive/";  #rclone挂载的本地文件夹，最后面保留/
LocalDIR="/data/";  #Aria2下载目录，最后面保留/

if [[ -z $(echo "$FileNum" |grep -o '[0-9]*' |head -n1) ]]; then FileNum='0'; fi
if [[ "$FileNum" -le '0' ]]; then exit 0; fi
if [[ "$#" != '3' ]]; then exit 0; fi

function LoadFile(){
  IFS_BAK=$IFS
  IFS=$'\n'
  if [[ ! -d "$LocalDIR" ]]; then return; fi
  if [[ -e "$File" ]]; then
    FileLoad="${File/#$LocalDIR}"
    while true
      do
        if [[ "$FileLoad" == '/' ]]; then return; fi
        echo "$FileLoad" |grep -q '/';
        if [[ "$?" == "0" ]]; then
          FileLoad=$(dirname "$FileLoad");
        else
          break;
        fi;
      done;
    if [[ "$FileLoad" == "$LocalDIR" ]]; then return; fi
    EXEC="$(command -v mv)"
    if [[ -z "$EXEC" ]]; then return; fi
    Option=" -f";
    cd "$LocalDIR";
    if [[ -e "$FileLoad" ]]; then
      ItemSize=$(du -s "$FileLoad" |cut -f1 |grep -o '[0-9]*' |head -n1)
      if [[ -z "$ItemSize" ]]; then return; fi
      if [[ "$ItemSize" -le "$MinSize" ]]; then
        echo -ne "\033[33m$FileLoad \033[0mtoo small to spik.\n";
        return;
      fi
      if [[ "$ItemSize" -ge "$MaxSize" ]]; then
        echo -ne "\033[33m$FileLoad \033[0mtoo large to spik.\n";
        return;
      fi
      eval "${EXEC}${Option}" \'"${FileLoad}"\' "${RemoteDIR}";
    fi
  fi
  IFS=$IFS_BAK
}
LoadFile;

make file become executable: chmod +x rcloneupload.sh

Edit Aria2 configuration file (/app/conf/aria2.conf) to add one following line at the file end:

on-download-complete=/app/conf/rcloneupload.sh

Restart aria2-ui docker to take change into effect

[root@centos-nextcloud-aria2 ~]# docker restart aria2-ui
bash-4.3# cd /app/conf/
bash-4.3# ls
aria2.conf      aria2.session      aria2c.sh      key

bash-4.3# vi /app/conf/aria2.conf
# Bit Torrent: The amount of time and the upload-to-download ratio you wish to
# seed to. If either the time limit ( seconds ) or the seed ratio is reached,
# torrent seeding will stop. You can set seed-time to zero(0) to disable
# seeding completely.
seed-ratio=0.01
seed-time=1
on-download-complete=/app/conf/rcloneupload.sh

Configure Host Rclone to mount Google Drive

1 First to install epel source

yum -y install epel-release

2  Install some components
yum -y install wget unzip screen fuse fuse-devel

3  Install rclone
[root@centos7-test1 data]# curl https://ift.tt/3p61n1s | sudo bash

4  configure rclone
rclone config

create a new remote and name it as google-drive

Other steps can check post: https://blog.51sec.org/2020/05/use-rclone-to-mount-google-drive-and.html

5  mount Google Drive using rclone

create a new folder at /home/gdrive

mkdir -p /home/gdrive

mount system

rclone mount google-drive: /home/gdrive --allow-other --allow-non-empty --vfs-cache-mode writes

Note: google-drive is the Rclone configuration name.

6 create rclone.service

To make rclone mount the google drive even after rebooted the vps, create /usr/lib/systemd/system/rclone.service with following information:
vi /usr/lib/systemd/system/rclone.service

[Unit]
Description=rclone

[Service]
User=root
ExecStart=/usr/bin/rclone mount google-drive: /home/gdrive --allow-other --allow-non-empty --vfs-cache-mode writes
Restart=on-abort

7 systemctl enable rclone.service

8 Using browser to access system

FileBrowser: http://ip/files
aria2ng: http://ip

Default FileBrowser username/password will be admin/admin.

Aria will download files to docker's local /data folder . Once completed downloading, the file will be moved to /gdrive folder.

Gdrive folder has been mounted with Google Drive, which will be managed by FileBrowser.

Cloudflare Configuration

Remove Installed Docker

#Delete Docker ContainerID=`docker ps|grep wahyd4/aria2-ui|awk '{print $1}'` docker kill ${ContainerID} docker rm ${ContainerID} docker rmi `docker images|grep wahyd4/aria2-ui|awk '{print $3}'` #Remove Download Folder rm -rf ~/data
rm -rf /home/gdrive

References

via Blogger https://ift.tt/3fxrZVZ
May 29, 2021 at 03:42PM Cloud, Docker

0 Comments

Build Confidence