Compare commits

...

34 Commits

Author SHA1 Message Date
b079a26969 add example from pre-build image 2023-07-14 19:41:32 +02:00
c2a6c66dad Change to debian slim for full support 2023-07-14 19:41:18 +02:00
5f13192006 Merge branch 'main' into tool_bandwhich 2023-07-14 19:40:52 +02:00
a3739784b5 Merge branch 'tool_synapse-compress-state' 2023-05-10 00:28:47 +02:00
b5c5ed1b12 Use pre-build image in example 2023-05-10 00:28:41 +02:00
8304f68e4e Fix env naming and location 2023-05-09 23:58:50 +02:00
5b7dbe1157 Merge branch 'tool_synapse-compress-state' 2023-05-09 23:52:21 +02:00
a760df23a7 Fix compile not working due to different folders 2023-05-09 23:51:44 +02:00
c99e9b07c2 Add example for using CLI 2023-05-09 22:20:39 +02:00
b0a6f904c9 Fix naming in docs 2023-05-09 22:20:28 +02:00
5b42ac9288 Merge branch 'tool_synapse-compress-state' 2023-04-05 20:36:26 +02:00
36aa46c486 Add documentation 2023-04-05 20:36:18 +02:00
4c06fe989a Merge branch 'main' into tool_synapse-compress-state 2023-04-05 20:32:30 +02:00
0837928200 Merge branch 'tool_auto-editor' 2023-04-05 20:30:09 +02:00
2c1756228f Add documentation 2023-04-05 20:28:45 +02:00
2baaf6f6fd Add workdir to release 2023-04-05 20:24:55 +02:00
9cca8016c4 First version of auto-editor 2023-04-04 13:42:29 +02:00
9960842516 Add instructions for synapse-compress 2023-04-04 11:15:59 +02:00
a62258447b Merge branch 'dev' 2022-11-24 09:53:56 +01:00
3ea86679d0 Merge branch 'tool_bandwhich' into dev 2022-11-24 09:53:45 +01:00
a5ef65e47f Switch script-header to bash 2022-11-23 20:03:37 +01:00
d6b3f93d72 Merge branch 'tool_bandwhich' into dev 2022-11-23 19:57:40 +01:00
4a0bc41f9b Merge branch 'main' into dev 2022-11-23 19:57:29 +01:00
72b0ef8b11 Merge branch 'dev' 2022-11-23 19:30:26 +01:00
3aeac81cf4 Fix wrong path in clone 2022-11-23 19:30:22 +01:00
62ee2fe065 Merge branch 'tool_bandwhich' 2022-11-23 19:24:44 +01:00
03622eaa69 Merge branch 'dev' 2022-11-23 19:22:53 +01:00
0354046e3c Add Instructions on how to use project 2022-11-23 19:09:11 +01:00
dfb6fc6870 Ignore build-folder 2022-11-23 19:01:36 +01:00
019a6d3f63 Move tag to build.sh 2022-11-23 18:57:24 +01:00
85a3dbfb96 Use Workdir-variable from arg 2022-11-23 18:38:52 +01:00
b6c8457bc3 Move args-declaration to build 2022-11-23 18:37:17 +01:00
d32f36dd29 Add latest-tag to image creation 2022-11-23 13:38:51 +01:00
ace27f7868 Fix default checkout 2022-11-23 13:20:52 +01:00
11 changed files with 191 additions and 24 deletions

1
.gitignore vendored Normal file
View File

@ -0,0 +1 @@
.build/

View File

@ -6,24 +6,108 @@ Collection of Dockerfiles created to build & pack various tools.
<br>
<!-- TOC -->
- [1. Tools](#1-tools)
- [1.1. Bandwhich](#11-bandwhich)
- [1. How to use](#1-how-to-use)
- [1.1. CLI](#11-cli)
- [2. Tools](#2-tools)
- [2.1. Bandwhich](#21-bandwhich)
- [2.2. auto-editor](#22-auto-editor)
- [2.3. synapse-compress-state](#23-synapse-compress-state)
<!-- /TOC -->
<br>
# 1. Tools
# 1. How to use
## 1.1. Bandwhich
Run the build-script `scripts/build.sh` with the path to the tool you want to build.
### 1.1.1. Source
<br>
The docker-image will be tagged with `default` and the detected version/commit-hash.
You can add more tags or change the existing tag-naming-scheme using the environment-variable `TAG`.
<br>
## 1.1. CLI
`scripts/build.sh <toolPath> [extra-args for docker build ..]`
<br>
### 1.1.1. Example
`scripts/build.sh tools/bandwhich/`
<br>
### 1.1.2. Environment-Variables
Variable | Description | Default
-|-|-
`TAG` | Tag for Docker-building | `TAGPREFIX` + `NAME`
`TAGPREFIX` | Set a prefix to the default-tag | -
`NAME` | Change the name used in the tag | *FolderName*
<br>
# 2. Tools
## 2.1. Bandwhich
### 2.1.1. Source
>This is a CLI utility for displaying current network utilization by process, connection and remote IP/hostname
https://github.com/imsnif/bandwhich
<br>
### 1.1.2. Example
### 2.1.2. Example
```
docker run -it --net host bandwhich -i eth0
docker run --rm -v /proc:/proc ghcr.io/ruakij/dockerbuilds/bandwhich:latest -n -i eth0
```
|||
-|-
`--rm` * | Removes the container when its stopped
`-v /proc:/proc` * | Makes Process identification from host-processes possible
`-n` * | Dont resolve IPs
`-i eth0` * | Only check on eth0 interface
*\* is Optional*
<br>
## 2.2. auto-editor
### 2.2.1. Source
>Auto-Editor is a command line application for automatically editing video and audio by analyzing a variety of methods, most notably audio loudness.
https://github.com/WyattBlue/auto-editor
<br>
### 2.2.2. Example
Speed-up silent part of videos:
```
docker run -v /home/ruakij/Videos:/video auto-build --no-open --silent-speed 8 --margin 0.2sec "input.webm" -o edited.mp4
```
<br>
### 2.2.3. Comment
TODO: Unfortunately the tool requires ffmpeg which is a huge tool and blows up the size of the image. I have to find a way to use a minimized version of it.
<br>
## 2.3. synapse-compress-state
Includes synapse_auto_compressor
### 2.3.1. Source
>This workspace contains experimental tools that attempt to reduce the number of rows in the state_groups_state table inside of a Synapse Postgresql database.
https://github.com/matrix-org/rust-synapse-compress-state
<br>
### 2.3.2. Example
```
docker run --net container:synapse ghcr.io/ruakij/dockerbuilds/synapse-compress-state:latest synapse_auto_compressor -p postgresql://synapse:synapse@127.0.0.1/synapse -c 500 -n 100
```

View File

@ -1,9 +1,21 @@
#!/bin/sh
#!/bin/bash
scriptPath_file=$(dirname "${BASH_SOURCE[0]}")
scriptPath_folder=$(realpath "${scriptPath_file}")
utils_path="${scriptPath_folder}/utils"
WORKDIR=${1:-$PWD}
WORKDIR=$(realpath "$WORKDIR")
# -- VARIABLES --
# Store original path
ORIGINALDIR=$PWD
# Get args without path
EXTRA_ARGS="${@:2}"
# Call prepare
printf "# PREPARE\n"
source ${utils_path}/prepare.sh
@ -12,3 +24,7 @@ source ${utils_path}/prepare.sh
printf "# BUILD\n"
build_type=${BUILD:-"ownarch"}
source ${utils_path}/build-${build_type}.sh
# Switch back to original path
cd $ORIGINALDIR

View File

@ -1,10 +1,9 @@
#!/bin/bash
TAG=${TAG:-"${TAG_PREFIX}${NAME}"}
PLATFORM=${PLATFORM:-"linux/amd64,linux/arm64/v8,linux/arm/v7"}
EXTRA_ARGS=${EXTRA_ARGS:-"$@"}
docker buildx build \
--platform $PLATFORM \
--tag $TAG:latest \
--tag "${TAG}:${VERSION}" \
$EXTRA_ARGS \
.
$WORKDIR

View File

@ -1,8 +1,7 @@
#!/bin/bash
TAG=${TAG:-"${TAG_PREFIX}${NAME}"}
EXTRA_ARGS=${EXTRA_ARGS:-"$@"}
docker build \
--tag $TAG:latest \
--tag $TAG:$VERSION \
$EXTRA_ARGS \
.
"$WORKDIR"

View File

@ -1,24 +1,33 @@
#!/bin/sh
#!/bin/bash
# --- VARIABLES ---
# Load variables
source $PWD/info.env
source $WORKDIR/info.env
# Check if certain vars are overwritten, set them if missing
if [ "${NAME}" = "" ]; then
NAME="$(basename $PWD)"
NAME="$(basename $WORKDIR)"
fi
export NAME
printf "Name: ${NAME}\n"
repositoryFolder="$WORKDIR/.build/repository"
# --- SOURCES ---
alias git="git -C .build/repository"
# Get sources locally
if ! [ -d .build/repository ]; then
git clone "${GIT_REPOSITORY}" .build/repository
else
git checkout - # Checkout previous branch in case GIT_CHECKOUT was different
if ! [ -d "$repositoryFolder" ]; then
git clone "${GIT_REPOSITORY}" "$repositoryFolder"
cloned=True
fi
# Change into repository
cd "$repositoryFolder"
if [ "$cloned" != True ]; then
# Checkout default-branch in case GIT_CHECKOUT was different
defaultBranch=$(git symbolic-ref refs/remotes/origin/HEAD | sed 's@^refs/remotes/origin/@@')
git checkout "${defaultBranch}"
git pull # Get changes
fi
@ -42,5 +51,5 @@ fi
export VERSION
printf "Version: ${VERSION}\n"
TAG=${TAG:-"${TAG_PREFIX}${NAME}"}
TAG=${TAG:-"${TAGPREFIX}${NAME}"}
printf "> ${TAG}\n"

View File

@ -0,0 +1,24 @@
# ---- Base ----
FROM python:slim AS base
#ENV PYTHONDONTWRITEBYTECODE=1
WORKDIR /app
# Install packages
RUN apt update && apt install -y ffmpeg && rm -rf /var/lib/apt/lists/*
#RUN pip install --no-compile --no-cache-dir ae-ffmpeg
#COPY .build/repository/ae-ffmpeg/ae_ffmpeg/Darwin-x86_64/* /usr/bin/
# ---- Build ----
FROM base AS build
ENV PYTHONDONTWRITEBYTECODE=1
# Copy sources
ADD .build/repository/auto_editor auto_editor
ADD .build/repository/setup.py .
ADD .build/repository/README.md .
# Run install
RUN pip install --no-compile --no-cache-dir -e .
# ---- Release ----
FROM build AS release
WORKDIR /videos
ENTRYPOINT ["auto-editor"]

View File

@ -0,0 +1,2 @@
GIT_REPOSITORY=https://github.com/WyattBlue/auto-editor
TAG=latest-tag

View File

@ -1,5 +1,5 @@
# ---- Base ----
FROM alpine:3 AS base
FROM debian:stable-slim AS base
WORKDIR /app

View File

@ -0,0 +1,32 @@
# ---- Base ----
FROM alpine AS base
WORKDIR /app
# Install dependencies
RUN apk add --no-cache libgcc libssl1.1 libc6-compat
# ---- Build ----
FROM rust:alpine3.16 AS build
WORKDIR /build
# Install packages
RUN apk add python3 musl-dev pkgconfig openssl-dev make
# Set build-flags
ENV RUSTFLAGS="-C target-feature=-crt-static"
# Copy sources
ADD .build/repository/ .
FROM build AS build-synapse_compress-state
# Update dependencies and Compile
RUN cargo update && cargo build --release
FROM build AS build-synapse_auto_compressor
# Update dependencies and Compile
RUN cd synapse_auto_compressor/ && cargo update && cargo build --release
# ---- Release ----
FROM base AS release
## Copy build-target
COPY --from=build-synapse_compress-state /build/target/release/synapse_compress_state .
COPY --from=build-synapse_auto_compressor /build/target/release/synapse_auto_compressor .
ENTRYPOINT ["./synapse_compress_state"]

View File

@ -0,0 +1 @@
GIT_REPOSITORY=https://github.com/matrix-org/rust-synapse-compress-state