matrix-spam-ml, branch HEAD

Issue bans and basic banlist support

2022-12-23T22:48:46Z

commit 99d2d7b015a8c7b47b56524c007058ef6cf98e0b parent 5ec4ab3fa9a81d73e323c552601288861d5c01c9 Author: MTRNord Date: Fri, 23 Dec 2022 23:48:46 +0100 Issue bans and basic banlist support

Update docs

2022-12-22T12:06:47Z

commit 5ec4ab3fa9a81d73e323c552601288861d5c01c9 parent 13db0175e2873d1ab9e46640a9a2629b107fff82 Author: MTRNord Date: Thu, 22 Dec 2022 14:06:47 +0200 Update docs

Update docs

2022-12-22T11:58:39Z

commit 13db0175e2873d1ab9e46640a9a2629b107fff82 parent 208ef5670c0110ce1c6b5d9b406614cd9a9f4679 Author: MTRNord Date: Thu, 22 Dec 2022 13:58:39 +0200 Update docs

Setup docs

2022-12-22T11:51:00Z

commit 208ef5670c0110ce1c6b5d9b406614cd9a9f4679 parent 6314c75c9f3b05fcdee3e17181520af21423dd4b Author: MTRNord Date: Thu, 22 Dec 2022 13:51:00 +0200 Setup docs

Also track original room id

2022-12-20T19:58:30Z

commit 6314c75c9f3b05fcdee3e17181520af21423dd4b parent 679a55d2801762092e0ddd54f108bfdf0828eb6b Author: MTRNord Date: Tue, 20 Dec 2022 20:58:30 +0100 Also track original room id

Add tracking info for the messages to make reaction handling easier

2022-12-20T19:03:06Z

commit 679a55d2801762092e0ddd54f108bfdf0828eb6b parent 13c0d60e7168d229d2a3323f61bdf66e0f4759e4 Author: MTRNord Date: Tue, 20 Dec 2022 20:03:06 +0100 Add tracking info for the messages to make reaction handling easier

Fix roomid pill

2022-12-20T18:37:49Z

commit 13c0d60e7168d229d2a3323f61bdf66e0f4759e4 parent 051e9debc6b0604a64e1dfd2f29a76c7c2498fc2 Author: MTRNord Date: Tue, 20 Dec 2022 19:37:49 +0100 Fix roomid pill

Add pills

2022-12-20T18:33:19Z

commit 051e9debc6b0604a64e1dfd2f29a76c7c2498fc2 parent 924a2bf858011a7cea9569d7169edf713db6ae28 Author: MTRNord Date: Tue, 20 Dec 2022 19:33:19 +0100 Add pills

Do some cleanup on the bot and prepare handling reactions

2022-12-20T18:10:45Z

commit 924a2bf858011a7cea9569d7169edf713db6ae28 parent 7f97efbe719b5fa08d0c6f104414dd0296f88e77 Author: MTRNord Date: Tue, 20 Dec 2022 19:10:45 +0100 Do some cleanup on the bot and prepare handling reactions

Add and apply eslint

2022-12-20T17:40:20Z

commit 7f97efbe719b5fa08d0c6f104414dd0296f88e77 parent 6accde390ef7ddf6a0e84ac1e2cba4ad244edb2d Author: MTRNord Date: Tue, 20 Dec 2022 18:40:20 +0100 Add and apply eslint

Cleanup bot and introduce basic warning formatting and rooms

2022-12-19T19:22:36Z

commit 6accde390ef7ddf6a0e84ac1e2cba4ad244edb2d parent 42d21bfc1aa224c74c695bcf2b736b9afa284a75 Author: MTRNord Date: Mon, 19 Dec 2022 20:22:36 +0100 Cleanup bot and introduce basic warning formatting and rooms

Update and rename LICENSE to LICENSE.md

2022-12-17T12:36:49Z

commit 42d21bfc1aa224c74c695bcf2b736b9afa284a75 parent d1cb4a4bcdaad059698695e3fbb71508b57fe619 Author: Marcel Date: Sat, 17 Dec 2022 13:36:49 +0100 Update and rename LICENSE to LICENSE.md Adds the commonclause https://commonsclause.com/

Don't do lfs

2022-12-10T22:18:07Z

commit d1cb4a4bcdaad059698695e3fbb71508b57fe619 parent be04cfcae1e47d99899932c288e85980a0cab8e2 Author: MTRNord Date: Sat, 10 Dec 2022 23:18:07 +0100 Don't do lfs

Remove lfs

2022-12-10T21:41:00Z

commit be04cfcae1e47d99899932c288e85980a0cab8e2 parent 45fab02c7db64ee4f92749aca91a4858f00c3e8e Author: MTRNord Date: Sat, 10 Dec 2022 22:41:00 +0100 Remove lfs

New bert model

2022-12-10T21:21:53Z

commit 45fab02c7db64ee4f92749aca91a4858f00c3e8e parent aa9282e06910c6d7d1bc0f6b71780ef23bd5a24d Author: MTRNord Date: Sat, 10 Dec 2022 22:21:53 +0100 New bert model

Remove dist folder

2022-12-10T19:18:46Z

commit aa9282e06910c6d7d1bc0f6b71780ef23bd5a24d parent df29ac59f4c7cf4da467b1755ca5d63e4b710cf1 Author: MTRNord Date: Sat, 10 Dec 2022 20:18:46 +0100 Remove dist folder

Add small bot

2022-12-10T18:02:35Z

commit df29ac59f4c7cf4da467b1755ca5d63e4b710cf1 parent 6328a8b0e4fbeb8798c08ea94c5c1d259d655fa3 Author: MTRNord Date: Sat, 10 Dec 2022 19:02:35 +0100 Add small bot

Add tests to bert model and print how many are correct in the end

2022-12-10T16:01:54Z

commit 6328a8b0e4fbeb8798c08ea94c5c1d259d655fa3 parent 500cd2923951e2a9ce670ad0eba0da06678b238b Author: MTRNord Date: Sat, 10 Dec 2022 17:01:54 +0100 Add tests to bert model and print how many are correct in the end

Add bert

2022-12-07T23:44:12Z

commit 500cd2923951e2a9ce670ad0eba0da06678b238b parent e15531675edfd2383c2a7003edc6936ae6469b8c Author: MTRNord Date: Thu, 8 Dec 2022 00:44:12 +0100 Add bert

Cleanup

2022-12-06T23:15:13Z

commit e15531675edfd2383c2a7003edc6936ae6469b8c parent 9e05eb8db3819cbc40ac59d639457e09c7a8047f Author: MTRNord Date: Wed, 7 Dec 2022 00:15:13 +0100 Cleanup

Fix input data

2022-12-06T23:13:23Z

commit 9e05eb8db3819cbc40ac59d639457e09c7a8047f parent 6d3c2290b23ce8e9c656dc8d0d4804d94060d3bf Author: MTRNord Date: Wed, 7 Dec 2022 00:13:23 +0100 Fix input data

Fix typo in model_v2.py

2022-12-06T23:08:19Z

commit 6d3c2290b23ce8e9c656dc8d0d4804d94060d3bf parent 94ca84b4cd7e3c4374f499a906a0bce4c0c338d7 Author: MTRNord Date: Wed, 7 Dec 2022 00:08:19 +0100 Fix typo in model_v2.py

Fix hyperparam training

2022-12-06T18:50:42Z

commit 94ca84b4cd7e3c4374f499a906a0bce4c0c338d7 parent ff58efb39835caea8c9f63529ad83a0f0603c291 Author: MTRNord Date: Tue, 6 Dec 2022 19:50:42 +0100 Fix hyperparam training

Apply clippy suggestion

2022-12-06T14:17:15Z

commit ff58efb39835caea8c9f63529ad83a0f0603c291 parent 9673c203424cfc80a0a455f6e618447ab0e4f477 Author: MTRNord Date: Tue, 6 Dec 2022 15:17:15 +0100 Apply clippy suggestion

Update webserver

2022-12-06T14:09:02Z

commit 9673c203424cfc80a0a455f6e618447ab0e4f477 parent 9f6768e90e60c95ee491673689928cacdf6a4bdd Author: MTRNord Date: Tue, 6 Dec 2022 15:09:02 +0100 Update webserver

Update rust deps

2022-11-23T20:45:18Z

commit 9f6768e90e60c95ee491673689928cacdf6a4bdd parent e779e49030581de0e8be670a95795bf9a4ddf27b Author: MTRNord Date: Wed, 23 Nov 2022 21:45:18 +0100 Update rust deps

Disable cargo-vet

2022-11-23T20:43:18Z

commit e779e49030581de0e8be670a95795bf9a4ddf27b parent f93e5aae40f6774a7942900098b21643d138d361 Author: MTRNord Date: Wed, 23 Nov 2022 21:43:18 +0100 Disable cargo-vet

Cleanup and fixes

2022-11-23T20:41:11Z

commit f93e5aae40f6774a7942900098b21643d138d361 parent 71d635f2bb7a89665a0e7f4964accd92393499fc Author: MTRNord Date: Wed, 23 Nov 2022 21:41:11 +0100 Cleanup and fixes

Fixup model py

2022-11-23T20:31:00Z

commit 71d635f2bb7a89665a0e7f4964accd92393499fc parent de1869929bb860e6ba00520bd523083d37319e10 Author: MTRNord Date: Wed, 23 Nov 2022 21:31:00 +0100 Fixup model py

run bert.ipynb and ignore conda env

2022-11-23T20:08:16Z

commit de1869929bb860e6ba00520bd523083d37319e10 parent 0de41d70bc4427bb5da19d05395a487a33363fb3 Author: MTRNord Date: Wed, 23 Nov 2022 21:08:16 +0100 run bert.ipynb and ignore conda env

Remove illegal quotes

2022-11-23T13:56:12Z

commit 0de41d70bc4427bb5da19d05395a487a33363fb3 parent da9b239f846857934afcd65bbee9804668512f7f Author: Marcel Date: Wed, 23 Nov 2022 14:56:12 +0100 Remove illegal quotes

Add more spam samples

2022-11-23T13:55:32Z

commit da9b239f846857934afcd65bbee9804668512f7f parent 3cd5efab3a87b71fb54f36c06bf70f6d0ced1498 Author: Marcel Date: Wed, 23 Nov 2022 14:55:32 +0100 Add more spam samples

Update README.md

2022-11-23T13:29:11Z

commit 3cd5efab3a87b71fb54f36c06bf70f6d0ced1498 parent 2ab9e33188ee3291ffeb7b5bd1b2607cb4606599 Author: Marcel Date: Wed, 23 Nov 2022 14:29:11 +0100 Update README.md

Fix listener?

2022-10-11T13:21:20Z

commit 2ab9e33188ee3291ffeb7b5bd1b2607cb4606599 parent 070af63942f7cf1c875befc5d5cf1e1fc466919b Author: MTRNord Date: Tue, 11 Oct 2022 15:21:20 +0200 Fix listener?

Copy templates over

2022-10-11T12:42:23Z

commit 070af63942f7cf1c875befc5d5cf1e1fc466919b parent b5617652498d2569c85eec446d9e705d069a1925 Author: MTRNord Date: Tue, 11 Oct 2022 14:42:23 +0200 Copy templates over

Copy templates to the main folder

2022-10-10T20:20:05Z

commit b5617652498d2569c85eec446d9e705d069a1925 parent b28c990616920d39c1899c59f6514a1856b95404 Author: MTRNord Date: Mon, 10 Oct 2022 22:20:05 +0200 Copy templates to the main folder

Log index access

2022-10-10T20:15:53Z

commit b28c990616920d39c1899c59f6514a1856b95404 parent cee430f270c4eeeb01eeadd7ea4768a47b0b3248 Author: MTRNord Date: Mon, 10 Oct 2022 22:15:53 +0200 Log index access

Add EXPOSE

2022-10-10T20:15:32Z

commit cee430f270c4eeeb01eeadd7ea4768a47b0b3248 parent da021d7650364cf6bfea734df21c1463695083db Author: MTRNord Date: Mon, 10 Oct 2022 22:15:32 +0200 Add EXPOSE

Listen on 0.0.0.0

2022-10-10T19:58:20Z

commit da021d7650364cf6bfea734df21c1463695083db parent f08a88cce2cceaaa50b94fc775268fcb44cc3344 Author: MTRNord Date: Mon, 10 Oct 2022 21:58:20 +0200 Listen on 0.0.0.0

Make sure that we actually lfs pull

2022-10-10T19:42:53Z

commit f08a88cce2cceaaa50b94fc775268fcb44cc3344 parent 7aefb5f41403cc08bedc42fb329633f36f92f629 Author: MTRNord Date: Mon, 10 Oct 2022 21:42:53 +0200 Make sure that we actually lfs pull

Run ldconfig

2022-10-10T19:38:25Z

commit 7aefb5f41403cc08bedc42fb329633f36f92f629 parent 3341284960ba72860c6a0cbdda242a96b789823b Author: MTRNord Date: Mon, 10 Oct 2022 21:38:25 +0200 Run ldconfig

Try a hack from https://github.com/kykosic/actix-tensorflow-example/blob/master/Dockerfile

2022-10-10T19:29:40Z

commit 3341284960ba72860c6a0cbdda242a96b789823b parent d2b5ee8927a11981ef03b7ee3a06d8f100cc3a9e Author: MTRNord Date: Mon, 10 Oct 2022 21:29:40 +0200 Try a hack from https://github.com/kykosic/actix-tensorflow-example/blob/master/Dockerfile

Debug why so file is missing

2022-10-10T19:27:40Z

commit d2b5ee8927a11981ef03b7ee3a06d8f100cc3a9e parent e829be92332ad66c75ecadf02d08e450e1d36f94 Author: MTRNord Date: Mon, 10 Oct 2022 21:27:40 +0200 Debug why so file is missing

Install teensorflow in docker

2022-10-10T19:12:16Z

commit e829be92332ad66c75ecadf02d08e450e1d36f94 parent 6871e75997f566bcd58dc18fa3902d89f4c31d4e Author: MTRNord Date: Mon, 10 Oct 2022 21:12:16 +0200 Install teensorflow in docker

Remove debug code

2022-10-10T19:01:35Z

commit 6871e75997f566bcd58dc18fa3902d89f4c31d4e parent 88d588d02323e4bc1571b98cec3805a4e1cb6fff Author: MTRNord Date: Mon, 10 Oct 2022 21:01:35 +0200 Remove debug code

also print pwd

2022-10-10T18:53:09Z

commit 88d588d02323e4bc1571b98cec3805a4e1cb6fff parent 3b63434ac559ef6558acd43ff2b17872ae378bb1 Author: MTRNord Date: Mon, 10 Oct 2022 20:53:09 +0200 also print pwd

Debug where the release is build in

2022-10-10T18:52:53Z

commit 3b63434ac559ef6558acd43ff2b17872ae378bb1 parent c8f3f6db2d8564a7240b4a75874f371aeb9e6ec9 Author: MTRNord Date: Mon, 10 Oct 2022 20:52:53 +0200 Debug where the release is build in

Fix docker image

2022-10-10T18:40:59Z

commit c8f3f6db2d8564a7240b4a75874f371aeb9e6ec9 parent da93da3719b2b2287d82e35db6bdd4679500b86e Author: MTRNord Date: Mon, 10 Oct 2022 20:40:59 +0200 Fix docker image

Berty the Bert

2022-10-10T17:58:50Z

commit da93da3719b2b2287d82e35db6bdd4679500b86e parent cf4d8b8e457e9636ac264cd700a9aeaa642a0113 Author: MTRNord Date: Mon, 10 Oct 2022 19:58:50 +0200 Berty the Bert

Move to a transfer training approach to not get issues due to the dataset. (hyperparameter search disabled as training it needs ~200-400GB which I currently dont have free)

2022-10-01T00:37:39Z

commit cf4d8b8e457e9636ac264cd700a9aeaa642a0113 parent a6bfcc4f3fcba60a559a19c22e0e7ffcf4bf7a49 Author: MTRNord Date: Sat, 1 Oct 2022 02:37:39 +0200 Move to a transfer training approach to not get issues due to the dataset. (hyperparameter search disabled as training it needs ~200-400GB which I currently dont have free)

Fix comment

2022-09-29T09:19:03Z

commit a6bfcc4f3fcba60a559a19c22e0e7ffcf4bf7a49 parent 8688eb71f48f22554fe2c880adf340cc24e8aee5 Author: Marcel Date: Thu, 29 Sep 2022 11:19:03 +0200 Fix comment

Mention that html should be stripped

2022-09-28T16:30:14Z

commit 8688eb71f48f22554fe2c880adf340cc24e8aee5 parent 5e8cf7790ca52ac0013344d460e709ada08279d1 Author: MTRNord Date: Wed, 28 Sep 2022 18:30:14 +0200 Mention that html should be stripped

Add new sample of spam and make sure that data send for review is sanitized

2022-09-28T16:29:03Z

commit 5e8cf7790ca52ac0013344d460e709ada08279d1 parent f5b0bb63ae37b1efb96c0edc85f8fabbca83355d Author: MTRNord Date: Wed, 28 Sep 2022 18:29:03 +0200 Add new sample of spam and make sure that data send for review is sanitized

Add cargo-vet

2022-09-28T13:52:09Z

commit f5b0bb63ae37b1efb96c0edc85f8fabbca83355d parent 5c4db680e9f82aedced3078605dd33fcb8c56bf2 Author: MTRNord Date: Wed, 28 Sep 2022 15:52:09 +0200 Add cargo-vet

Fix readme float and generate json_schema

2022-09-28T10:23:16Z

commit 5c4db680e9f82aedced3078605dd33fcb8c56bf2 parent 64b9f77b9e03f58782caf018d54ae9f0f76cc714 Author: MTRNord Date: Wed, 28 Sep 2022 12:23:16 +0200 Fix readme float and generate json_schema

Remove wandb. Newer model is seemingly worse than the previous one

2022-09-27T23:20:33Z

commit 64b9f77b9e03f58782caf018d54ae9f0f76cc714 parent 294d1ffc3d30c1c2f842eec9c9c32f1048a386ba Author: MTRNord Date: Wed, 28 Sep 2022 01:20:33 +0200 Remove wandb. Newer model is seemingly worse than the previous one

Fix COPY in Dockerfile

2022-09-27T18:40:36Z

commit 294d1ffc3d30c1c2f842eec9c9c32f1048a386ba parent 422c7c613c3aa1349e755f42970d552c10b66a04 Author: MTRNord Date: Tue, 27 Sep 2022 20:40:36 +0200 Fix COPY in Dockerfile

Add readme and fix image name

2022-09-27T18:38:57Z

commit 422c7c613c3aa1349e755f42970d552c10b66a04 parent 805ee41e4368c8bb3043b84d967e885855a14f4a Author: MTRNord Date: Tue, 27 Sep 2022 20:38:57 +0200 Add readme and fix image name

Use a rust server to serve the model via a highlevel api

2022-09-27T18:30:38Z

commit 805ee41e4368c8bb3043b84d967e885855a14f4a parent 460b51bca7f44c113ae61709d6ab4bdb2c26dca3 Author: MTRNord Date: Tue, 27 Sep 2022 20:30:38 +0200 Use a rust server to serve the model via a highlevel api

Improve building in v2.py

2022-09-27T06:02:20Z

commit 460b51bca7f44c113ae61709d6ab4bdb2c26dca3 parent 8007635cfc785c00bcf471202b09dfb0f272b318 Author: MTRNord Date: Tue, 27 Sep 2022 08:02:20 +0200 Improve building in v2.py

Update gitignore

2022-09-26T19:41:04Z

commit 8007635cfc785c00bcf471202b09dfb0f272b318 parent 3542d54dc3aed97f1d148b2672ff99a97c239deb Author: MTRNord Date: Mon, 26 Sep 2022 21:41:04 +0200 Update gitignore

Add nicer python script

2022-09-26T19:40:52Z

commit 3542d54dc3aed97f1d148b2672ff99a97c239deb parent 70fd7d46e4cfa7f5393286ceb7576fe91d8253df Author: MTRNord Date: Mon, 26 Sep 2022 21:40:52 +0200 Add nicer python script

Move to python script

2022-09-26T14:08:07Z

commit 70fd7d46e4cfa7f5393286ceb7576fe91d8253df parent a9f00dc8a5c7893996c1545edeacab87b258c96d Author: MTRNord Date: Mon, 26 Sep 2022 16:08:07 +0200 Move to python script

Another run (not useful)

2022-09-25T21:59:23Z

commit a9f00dc8a5c7893996c1545edeacab87b258c96d parent 15853a7c0ae15be927dc9c3ac74acfa6cc3f9895 Author: MTRNord Date: Sun, 25 Sep 2022 23:59:23 +0200 Another run (not useful)

Add hypervariables

2022-09-25T20:18:22Z

commit 15853a7c0ae15be927dc9c3ac74acfa6cc3f9895 parent 56bc99b8341638ce5b7dd080cd8923ace8990ab6 Author: MTRNord Date: Sun, 25 Sep 2022 22:18:22 +0200 Add hypervariables

Update model

2022-09-25T15:00:40Z

commit 56bc99b8341638ce5b7dd080cd8923ace8990ab6 parent 67602639ae9cc4f8107178ef338b3eebc8a8ac28 Author: MTRNord Date: Sun, 25 Sep 2022 17:00:40 +0200 Update model

Add new models and training data

2022-09-25T14:01:52Z

commit 67602639ae9cc4f8107178ef338b3eebc8a8ac28 parent cbb0817428b90522255712aaf9b4f2ac6c9ab67d Author: MTRNord Date: Sun, 25 Sep 2022 16:01:52 +0200 Add new models and training data

Run various new models

2022-09-25T13:48:25Z

commit cbb0817428b90522255712aaf9b4f2ac6c9ab67d parent 76129919574b62d05af23cecedf6c7acee598df4 Author: MTRNord Date: Sun, 25 Sep 2022 15:48:25 +0200 Run various new models

Create LICENSE

2022-09-25T01:14:50Z

commit 76129919574b62d05af23cecedf6c7acee598df4 parent 03a67fd48de8facfd68fa0503f88cfb443c007cf Author: Marcel Date: Sun, 25 Sep 2022 03:14:50 +0200 Create LICENSE

remove max length and set higher dim

2022-09-25T01:12:16Z

commit 03a67fd48de8facfd68fa0503f88cfb443c007cf parent b680b806791a21e8d555c53b1d1e2d64bb627679 Author: MTRNord Date: Sun, 25 Sep 2022 03:12:16 +0200 remove max length and set higher dim

Find the epoch sweetspot

2022-09-25T00:50:13Z

commit b680b806791a21e8d555c53b1d1e2d64bb627679 parent 1d79956eaccb660a8388c97a8eb97a338bf40759 Author: MTRNord Date: Sun, 25 Sep 2022 02:50:13 +0200 Find the epoch sweetspot

Try increasing max_length and fix gitignore

2022-09-25T00:27:22Z

commit 1d79956eaccb660a8388c97a8eb97a338bf40759 parent 4cdae5d45dfb652479e5e61e700d8975d9b31e84 Author: MTRNord Date: Sun, 25 Sep 2022 02:27:22 +0200 Try increasing max_length and fix gitignore

Try smaller max_length and remove checkpoints

2022-09-25T00:19:31Z

commit 4cdae5d45dfb652479e5e61e700d8975d9b31e84 parent a0aadfab646133808bf200df2eaa53b9c6107739 Author: MTRNord Date: Sun, 25 Sep 2022 02:19:31 +0200 Try smaller max_length and remove checkpoints

Initial experiments

2022-09-25T00:12:42Z

commit a0aadfab646133808bf200df2eaa53b9c6107739 Author: MTRNord Date: Sun, 25 Sep 2022 02:12:42 +0200 Initial experiments