Merge branch 'v4' of https://gitlab.com/kwoth/nadekobot into v4

Replaced all usages of ConcurrentDictionary with NonBlocking.ConcurrentDictionary

See merge request Kwoth/nadekobot!256

.gitlab-ci.yml

@@ -113,6 +113,7 @@ docker-build:
   # Use the official docker image.
   image: docker:latest
   stage: build
+  allow_failure: true
   services:
     - docker:dind
   before_script:

CHANGELOG.md

@@ -3,6 +3,59 @@
 
 Experimental changelog. Mostly based on [keepachangelog](https://keepachangelog.com/en/1.0.0/) except date format. a-c-f-r-o
 
+## Unreleased
+
+## [4.2.7] - 24.06.2022
+
+### Changed
+
+- New cache abstraction added
+  - 2 implemenations: redis and memory
+  - All current bots will stay on redis cache, all new bots will use **in-process memory cache by default**
+  - This change removes bot's hard dependency on redis
+  - Configurable in `creds.yml` (please read the comments)
+  - You **MUST** use 'redis' if your bot runs on more than 1 shard (2000+ servers)
+- [dev] Using new non-locking ConcurrentDictionary
+
+### Fixed
+
+- `.xp` will now show default user avatars too
+
+### Removed
+
+- Removed `.imagesreload` as images are now lazily loaded on request and then cached
+
+## [4.2.6] - 22.06.2022
+
+### Fixed
+
+- Patron system should now properly by disabled on selfhosts by default.
+
+## [4.2.5] - 18.06.2022
+
+### Fixed
+
+- Fixed `.crypto`, you will still need coinmarketcapApiKey in `creds.yml` in order to make it run consistently as the key is shared
+
+## [4.2.3] - 17.06.2022
+
+### Fixed
+
+- Fixed `.timely` nullref bug and made it nicer
+- Fixed `.streamrole` not updating in real time!
+- Disabling specific Global Expressions should now work with `.sc` (and other permission commands)
+
+## [4.2.2] - 15.06.2022
+
+### Fixed
+
+- Added missing Patron Tiers and fixed Patron pledge update bugs
+- Prevented creds_example.yml error in docker containers from crashing it
+
+### Changed
+
+- Rss feeds will now show error counter before deletion
+
 ## [4.2.1] - 14.06.2022
 
 ### Added
@@ -130,6 +183,7 @@ Note: Results of each `.youtube` query will be cached for 1 hour to improve perf
  
 - `.rh` no longer needs quotes for multi word roles
 - `.deletexp` will now properly delete server xp too
+- Fixed `.crypto` sparklines
 - [dev] added support for configs to properly parse enums without case sensitivity (ConfigParsers.InsensitiveEnum)
 - [dev] Fixed a bug in .gencmdlist  
 - [dev] small fixes to creds provider  
@@ -139,15 +193,6 @@ Note: Results of each `.youtube` query will be cached for 1 hour to improve perf
 - `.ddg` removed.
 - [dev] removed some dead code and comments
 
-### Obsolete
-
-
-  
-
-### Fixed
-
-- Fixed `.crypto` sparklines
-
 ## [4.1.6] - 14.05.2022
 
 ### Fixed

docs/custom-reactions.md

@@ -3,19 +3,19 @@
 ### Important
 
 - For modifying **global** expressions, the ones which will work across all the servers your bot is connected to, you **must** be a Bot Owner.  
-  You must also use the commands for adding, deleting and listing these reactions in a direct message with the bot.
+  You must also use the commands for adding, deleting and listing these expressions in a direct message with the bot.
 - For modifying **local** expressions, the ones which will only work on the server that they are added on, it is required to have the **Administrator** permission.  
-  You must also use the commands for adding, deleting and listing these reactions in the server you want the expressions to work on.
+  You must also use the commands for adding, deleting and listing these expressions in the server you want the expressions to work on.
 
 ### Commands and Their Use
 
 | Command Name | Description                                                                                                                                                                                                                                                                                | Example                          |
 | :----------: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------- |
-| `.exadd`  | Add an expression with a trigger and a response. Running this command in a server requries the Administrator permission. Running this command in DM is Bot Owner only, and adds a new global expression.             | `.exadd "hello" Hi there, %user%!` |
-|   `exl`   | Lists a page of global or server expression(15 reactions / expressions per page). Running this command in a DM will list the global expression, while running it in a server will list that server's expression.   | `.exl 1`                           |
+| `.exa`  | Add an expression with a trigger and a response. Running this command in a server requries the Administrator permission. Running this command in DM is Bot Owner only, and adds a new global expression.             | `.exadd "hello" Hi there, %user%!` |
+|   `exl`   | Lists a page of global or server expression(15 expressions per page). Running this command in a DM will list the global expression, while running it in a server will list that server's expression.   | `.exl 1`                           |
 |   `.exd`  | Deletes an expression based on the provided index. Running this command in a server requires the Administrator permission. Running this command in DM is Bot Owner only, and will delete a global expression.     | `.exd 5`                           |
 
-#### Now that we know the commands let's take a look at an example of adding a command with `.acr`,
+#### Now that we know the commands let's take a look at an example of adding a command with `.exa`,
 
 `.exadd "Nice Weather" It sure is, %user%!`
 
@@ -35,7 +35,7 @@ Now, if that command was ran in a server, anyone on that server can make the bot
 If you want to disable a global expression which you do not like, and you do not want to remove it, or you are not the bot owner, you can do so by adding a new expression with the same trigger on your server, and set the response to `-`.
 
 For example:
-`.acr /o/ -`
+`.exa /o/ -`
 
 Now if you try to trigger `/o/`, it won't print anything even if there is a global expression with the same name.
 

docs/guides/linux-guide.md

@@ -17,7 +17,7 @@ It is recommended that you use **Ubuntu 20.04**, as there have been nearly no pr
 
 ##### Compatible operating systems:
 
-- Ubuntu: 16.04, 18.04, 20.04, 21.04, 21.10
+- Ubuntu: 16.04, 18.04, 20.04, 21.04, 21.10 22.04
 - Mint: 19, 20
 - Debian: 9, 10
 - CentOS: 7
@@ -63,9 +63,20 @@ Open Terminal (if you're on an installation with a window manager) and navigate
 4. Run the bot (type `3` and press enter)
 5. 🎉 
 
+## **⚠ IF YOU ARE FOLLOWING THE GUIDE ABOVE, IGNORE THIS SECTION ⚠** 
+
 ## Linux Release
 
-**⚠ IF YOU ARE FOLLOWING THE GUIDE ABOVE, IGNORE THIS SECTION ⚠** 
+###### Prerequisites
+
+1. Nadeko requires redis to function
+    - ubuntu installation command: `sudo apt-get install redis-server`
+2. Playing music requires `ffmpeg`, `libopus`, `libsodium` and `youtube-dl` (which in turn requires python3)
+    - ubuntu installation command: `sudo apt-get install ffmpeg libopus0 opus-tools libopus-dev libsodium-dev -y`
+3. Make sure your python is version 3+ with `python --version`
+    - if it's not, you can install python 3 and make it the default with: `sudo apt-get install python3.8 python-is-python3`
+
+*You can use nadeko bash script [prerequisites installer](https://gitlab.com/Kwoth/nadeko-bash-installer/-/blob/v4/n-prereq.sh) as a reference*
 
 ##### Installation Instructions
 
@@ -92,19 +103,6 @@ Open Terminal (if you're on an installation with a window manager) and navigate
 
 ##### Release Update Instructions
 
-###### Prerequisites
-
-1. Nadeko requires redis to function
-    - ubuntu installation command: `sudo apt-get install redis-server`
-2. Playing music requires `ffmpeg`, `libopus`, `libsodium` and `youtube-dl` (which in turn requires python3)
-    - ubuntu installation command: `sudo apt-get install ffmpeg libopus0 opus-tools libopus-dev libsodium-dev -y`
-3. Make sure your python is version 3+ with `python --version`
-    - if it's not, you can install python 3 and make it the default with: `sudo apt-get install python3.8 python-is-python3`
-
-*You can use nadeko bash script [prerequisites installer](https://gitlab.com/Kwoth/nadeko-bash-installer/-/blob/v4/n-prereq.sh) as a reference*
-
-###### Installation
-
 1. Stop the bot
 2. Download the latest release from <https://gitlab.com/Kwoth/nadekobot/-/releases>
     - Look for the file called "x.x.x-linux-x64-build.tar" (where `X.X.X` is a version, for example 3.0.4) and download it

docs/guides/windows-guide.md

@@ -63,9 +63,9 @@ You can still install them manually:
 - [ffmpeg-32bit] | [ffmpeg-64bit] - Download the **appropriate version** for your system (32 bit if you're running a 32 bit OS, or 64 if you're running a 64bit OS). Unzip it, and move `ffmpeg.exe` to a path that's in your PATH environment variable. If you don't know what that is, then just move the `ffmpeg.exe` file to NadekoBot/system
 - [youtube-dl] - Click to download the file. Then put `youtube-dl.exe` in a path that's in your PATH environment variable. If you don't know what that is, then just move the `youtube-dl.exe` file to NadekoBot/system
 
-### Windows From Source
+## **⚠ IF YOU ARE FOLLOWING THE GUIDE ABOVE, IGNORE THIS SECTION ⚠**
 
-⚠ IF YOU ARE FOLLOWING THE GUIDE ABOVE, IGNORE THIS SECTION ⚠ 
+### Windows From Source
 
 ##### Prerequisites
 

docs/permissions-system.md

@@ -77,8 +77,8 @@ Say you want to only enable NSFW commands for a specific role, just do the follo
 
 If you don't want server or global Expressions, just block the module that controls their usage:
 
-1. `.sm Expressions disable`
-    - Disables the ActualCustomReactions module from being used
+1. `.sm ActualExpressions disable`
+    - Disables the ActualExpression module from being used
 
 **Note**: The `Expressions` module controls the usage of Expressions. The `Expressions` module controls commands related to Expressions (such as `.acr`, `.lcr`, `.crca`, etc).
 

mkdocs.yml

@@ -92,4 +92,3 @@ nav:
             - medusa/snek-lifecycle.md
     - Contribution Guide: contribution-guide.md
     - Donate: donate.md
-    - License: license.md

src/NadekoBot.Tests/BotStringsTests.cs

@@ -21,7 +21,7 @@ namespace NadekoBot.Tests
             var stringsSource = new LocalFileStringsSource(
                 responsesPath,
                 commandsPath);
-            var strings = new LocalBotStringsProvider(stringsSource);
+            var strings = new MemoryBotStringsProvider(stringsSource);
 
             var culture = new CultureInfo("en-US");
 

src/NadekoBot.Tests/ConcurrentHashSetTests.cs

@@ -0,0 +1,93 @@
+using System.Collections.Generic;
+using NUnit.Framework;
+
+namespace NadekoBot.Tests;
+
+public class ConcurrentHashSetTests
+{
+    private ConcurrentHashSet<(int?, int?)> _set;
+
+    [SetUp]
+    public void SetUp()
+    {
+        _set = new();
+    }
+
+    [Test]
+    public void AddTest()
+    {
+        var result = _set.Add((1, 2));
+        
+        Assert.AreEqual(true, result);
+        
+        result = _set.Add((1, 2));
+        
+        Assert.AreEqual(false, result);
+    }
+
+    [Test]
+    public void TryRemoveTest()
+    {
+        _set.Add((1, 2));
+        var result = _set.TryRemove((1, 2));
+        
+        Assert.AreEqual(true, result);
+        
+        result = _set.TryRemove((1, 2));
+        Assert.AreEqual(false, result);
+    }
+
+    [Test]
+    public void CountTest()
+    {
+        _set.Add((1, 2)); // 1
+        _set.Add((1, 2)); // 1
+        
+        _set.Add((2, 2)); // 2
+        
+        _set.Add((3, 2)); // 3
+        _set.Add((3, 2)); // 3
+        
+        Assert.AreEqual(3, _set.Count);
+    }
+
+    [Test]
+    public void ClearTest()
+    {
+        _set.Add((1, 2));
+        _set.Add((1, 3));
+        _set.Add((1, 4));
+        
+        _set.Clear();
+        
+        Assert.AreEqual(0, _set.Count);
+    }
+
+    [Test]
+    public void ContainsTest()
+    {
+        _set.Add((1, 2));
+        _set.Add((3, 2));
+        
+        Assert.AreEqual(true, _set.Contains((1, 2)));
+        Assert.AreEqual(true, _set.Contains((3, 2)));
+        Assert.AreEqual(false, _set.Contains((2, 1)));
+        Assert.AreEqual(false, _set.Contains((2, 3)));
+    }
+
+    [Test]
+    public void RemoveWhereTest()
+    {
+        _set.Add((1, 2));
+        _set.Add((1, 3));
+        _set.Add((1, 4));
+        _set.Add((2, 5));
+        
+        // remove tuples which have even second item
+        _set.RemoveWhere(static x => x.Item2 % 2 == 0);
+        
+        Assert.AreEqual(2, _set.Count);
+        Assert.AreEqual(true, _set.Contains((1, 3)));
+        Assert.AreEqual(true, _set.Contains((2, 5)));
+    }
+}

src/NadekoBot/Bot.cs

@@ -102,20 +102,20 @@ public sealed class Bot
         var svcs = new ServiceCollection().AddTransient(_ => _credsProvider.GetCreds()) // bot creds
                                           .AddSingleton(_credsProvider)
                                           .AddSingleton(_db) // database
-                                          .AddRedis(_creds.RedisOptions) // redis
                                           .AddSingleton(Client) // discord socket client
                                           .AddSingleton(_commandService)
                                           // .AddSingleton(_interactionService)
                                           .AddSingleton(this)
                                           .AddSingleton<ISeria, JsonSeria>()
-                                          .AddSingleton<IPubSub, RedisPubSub>()
                                           .AddSingleton<IConfigSeria, YamlSeria>()
-                                          .AddBotStringsServices(_creds.TotalShards)
                                           .AddConfigServices()
                                           .AddConfigMigrators()
                                           .AddMemoryCache()
                                           // music
-                                          .AddMusic();
+                                          .AddMusic()
+                                          // cache
+                                          .AddCache(_creds);
+        
         // admin
 #if GLOBAL_NADEKO
         svcs.AddSingleton<ILogCommandService, DummyLogCommandService>();
@@ -143,13 +143,6 @@ public sealed class Bot
                 .AddSingleton<IReadyExecutor>(x => x.GetRequiredService<RemoteGrpcCoordinator>());
         }
 
-        svcs.AddSingleton<RedisLocalDataCache>()
-            .AddSingleton<ILocalDataCache>(x => x.GetRequiredService<RedisLocalDataCache>())
-            .AddSingleton<RedisImagesCache>()
-            .AddSingleton<IImageCache>(x => x.GetRequiredService<RedisImagesCache>())
-            .AddSingleton<IReadyExecutor>(x => x.GetRequiredService<RedisImagesCache>())
-            .AddSingleton<IDataCache, RedisCache>();
-
         svcs.Scan(scan => scan.FromAssemblyOf<IReadyExecutor>()
                               .AddClasses(classes => classes.AssignableToAny(
                                       // services

src/NadekoBot/Common/Attributes/Ratelimit.cs

@@ -15,22 +15,24 @@ public sealed class RatelimitAttribute : PreconditionAttribute
         Seconds = seconds;
     }
 
-    public override Task<PreconditionResult> CheckPermissionsAsync(
+    public override async Task<PreconditionResult> CheckPermissionsAsync(
         ICommandContext context,
         CommandInfo command,
         IServiceProvider services)
     {
         if (Seconds == 0)
-            return Task.FromResult(PreconditionResult.FromSuccess());
+            return PreconditionResult.FromSuccess();
 
-        var cache = services.GetRequiredService<IDataCache>();
-        var rem = cache.TryAddRatelimit(context.User.Id, command.Name, Seconds);
+        var cache = services.GetRequiredService<IBotCache>();
+        var rem = await cache.GetRatelimitAsync(
+            new($"precondition:{context.User.Id}:{command.Name}"),
+            Seconds.Seconds());
 
         if (rem is null)
-            return Task.FromResult(PreconditionResult.FromSuccess());
+            return PreconditionResult.FromSuccess();
 
         var msgContent = $"You can use this command again in {rem.Value.TotalSeconds:F1}s.";
 
-        return Task.FromResult(PreconditionResult.FromError(msgContent));
+        return PreconditionResult.FromError(msgContent);
     }
 }

src/NadekoBot/Common/Cache/BotCacheExtensions.cs

@@ -0,0 +1,46 @@
+using OneOf;
+using OneOf.Types;
+
+namespace NadekoBot.Common;
+
+public static class BotCacheExtensions
+{
+    public static async ValueTask<T?> GetOrDefaultAsync<T>(this IBotCache cache, TypedKey<T> key)
+    {
+        var result = await cache.GetAsync(key);
+        if (result.TryGetValue(out var val))
+            return val;
+
+        return default;
+    }
+    
+    private static TypedKey<byte[]> GetImgKey(Uri uri)
+        => new($"image:{uri}");
+
+    public static ValueTask SetImageDataAsync(this IBotCache c, string key, byte[] data)
+        => c.SetImageDataAsync(new Uri(key), data);
+    public static async ValueTask SetImageDataAsync(this IBotCache c, Uri key, byte[] data)
+        => await c.AddAsync(GetImgKey(key), data, expiry: TimeSpan.FromHours(48));
+
+    public static async ValueTask<OneOf<byte[], None>> GetImageDataAsync(this IBotCache c, Uri key)
+        => await c.GetAsync(GetImgKey(key));
+
+    public static async Task<TimeSpan?> GetRatelimitAsync(
+        this IBotCache c,
+        TypedKey<long> key,
+        TimeSpan length)
+    {
+        var now = DateTime.UtcNow;
+        var nowB = now.ToBinary();
+
+        var cachedValue = await c.GetOrAddAsync(key,
+            () => Task.FromResult(now.ToBinary()),
+            expiry: length);
+
+        if (cachedValue == nowB)
+            return null;
+
+        var diff = now - DateTime.FromBinary(cachedValue);
+        return length - diff;
+    }
+}

src/NadekoBot/Common/Cache/IBotCache.cs

@@ -0,0 +1,47 @@
+using OneOf;
+using OneOf.Types;
+
+namespace NadekoBot.Common;
+
+public interface IBotCache
+{
+    /// <summary>
+    /// Adds an item to the cache
+    /// </summary>
+    /// <param name="key">Key to add</param>
+    /// <param name="value">Value to add to the cache</param>
+    /// <param name="expiry">Optional expiry</param>
+    /// <param name="overwrite">Whether old value should be overwritten</param>
+    /// <typeparam name="T">Type of the value</typeparam>
+    /// <returns>Returns whether add was sucessful. Always true unless ovewrite = false</returns>
+    ValueTask<bool> AddAsync<T>(TypedKey<T> key, T value, TimeSpan? expiry = null, bool overwrite = true);
+    
+    /// <summary>
+    /// Get an element from the cache
+    /// </summary>
+    /// <param name="key">Key</param>
+    /// <typeparam name="T">Type of the value</typeparam>
+    /// <returns>Either a value or <see cref="None"/></returns>
+    ValueTask<OneOf<T, None>> GetAsync<T>(TypedKey<T> key);
+    
+    /// <summary>
+    /// Remove a key from the cache
+    /// </summary>
+    /// <param name="key">Key to remove</param>
+    /// <typeparam name="T">Type of the value</typeparam>
+    /// <returns>Whether there was item</returns>
+    ValueTask<bool> RemoveAsync<T>(TypedKey<T> key);
+
+    /// <summary>
+    /// Get the key if it exists or add a new one
+    /// </summary>
+    /// <param name="key">Key to get and potentially add</param>
+    /// <param name="createFactory">Value creation factory</param>
+    /// <param name="expiry">Optional expiry</param>
+    /// <typeparam name="T">Type of the value</typeparam>
+    /// <returns>The retrieved or newly added value</returns>
+    ValueTask<T?> GetOrAddAsync<T>(
+        TypedKey<T> key,
+        Func<Task<T?>> createFactory,
+        TimeSpan? expiry = null);
+}

src/NadekoBot/Common/Cache/MemoryBotCache.cs

@@ -0,0 +1,71 @@
+using Microsoft.Extensions.Caching.Memory;
+using OneOf;
+using OneOf.Types;
+
+// ReSharper disable InconsistentlySynchronizedField
+
+namespace NadekoBot.Common;
+
+public sealed class MemoryBotCache : IBotCache
+{
+    // needed for overwrites and Delete return value
+    private readonly object _cacheLock = new object();
+    private readonly MemoryCache _cache;
+
+    public MemoryBotCache()
+    {
+        _cache = new MemoryCache(new MemoryCacheOptions());
+    }
+
+    public ValueTask<bool> AddAsync<T>(TypedKey<T> key, T value, TimeSpan? expiry = null, bool overwrite = true)
+    {
+        if (overwrite)
+        {
+            using var item = _cache.CreateEntry(key.Key);
+            item.Value = value;
+            item.AbsoluteExpirationRelativeToNow = expiry;
+            return new(true);
+        }
+        
+        lock (_cacheLock)
+        {
+            if (_cache.TryGetValue(key.Key, out var old) && old is not null)
+                return new(false);
+            
+            using var item = _cache.CreateEntry(key.Key);
+            item.Value = value;
+            item.AbsoluteExpirationRelativeToNow = expiry;
+            return new(true);
+        }
+    }
+
+    public async ValueTask<T?> GetOrAddAsync<T>(
+        TypedKey<T> key,
+        Func<Task<T?>> createFactory,
+        TimeSpan? expiry = null)
+        => await _cache.GetOrCreateAsync(key.Key,
+            async ce =>
+            {
+                ce.AbsoluteExpirationRelativeToNow = expiry;
+                var val = await createFactory();
+                return val;
+            });
+
+    public ValueTask<OneOf<T, None>> GetAsync<T>(TypedKey<T> key)
+    {
+        if (!_cache.TryGetValue(key.Key, out var val) || val is null)
+            return new(new None());
+
+        return new((T)val);
+    }
+
+    public ValueTask<bool> RemoveAsync<T>(TypedKey<T> key)
+    {
+        lock (_cacheLock)
+        {
+            var toReturn = _cache.TryGetValue(key.Key, out var old ) && old is not null;
+            _cache.Remove(key.Key);
+            return new(toReturn);
+        }
+    }
+}

src/NadekoBot/Common/Cache/RedisBotCache.cs

@@ -0,0 +1,119 @@
+using OneOf;
+using OneOf.Types;
+using StackExchange.Redis;
+using System.Text.Json;
+using System.Text.Json.Serialization;
+
+namespace NadekoBot.Common;
+
+public sealed class RedisBotCache : IBotCache
+{
+    private static readonly Type[] _supportedTypes = new []
+    {
+        typeof(bool), typeof(int), typeof(uint), typeof(long),
+        typeof(ulong), typeof(float), typeof(double),
+        typeof(string), typeof(byte[]), typeof(ReadOnlyMemory<byte>), typeof(Memory<byte>),
+        typeof(RedisValue),
+    };
+
+    private static readonly JsonSerializerOptions _opts = new()
+    {
+        PropertyNameCaseInsensitive = true,
+        NumberHandling = JsonNumberHandling.AllowReadingFromString,
+        AllowTrailingCommas = true,
+        IgnoreReadOnlyProperties = false,
+    };
+    private readonly ConnectionMultiplexer _conn;
+
+    public RedisBotCache(ConnectionMultiplexer conn)
+    {
+        _conn = conn;
+    }
+
+    public async ValueTask<bool> AddAsync<T>(TypedKey<T> key, T value, TimeSpan? expiry = null, bool overwrite = true)
+    {
+        // if a null value is passed, remove the key
+        if (value is null)
+        {
+            await RemoveAsync(key);
+            return false;
+        }
+        
+        var db = _conn.GetDatabase();
+        RedisValue val = IsSupportedType(typeof(T)) 
+            ? RedisValue.Unbox(value)
+            : JsonSerializer.Serialize(value, _opts);
+
+        var success = await db.StringSetAsync(key.Key,
+            val,
+            expiry: expiry,
+            when: overwrite ? When.Always : When.NotExists);
+
+        return success;
+    }
+
+    public bool IsSupportedType(Type type)
+    {
+        if (type.IsGenericType)
+        {
+            var typeDef = type.GetGenericTypeDefinition();
+            if (typeDef == typeof(Nullable<>))
+                return IsSupportedType(type.GenericTypeArguments[0]);
+        }
+
+        foreach (var t in _supportedTypes)
+        {
+            if (type == t)
+                return true;
+        }
+
+        return false;
+    }
+    
+    public async ValueTask<OneOf<T, None>> GetAsync<T>(TypedKey<T> key)
+    {
+        var db = _conn.GetDatabase();
+        var val = await db.StringGetAsync(key.Key);
+        if (val == default)
+            return new None();
+
+        if (IsSupportedType(typeof(T)))
+            return (T)((IConvertible)val).ToType(typeof(T), null);
+
+        return JsonSerializer.Deserialize<T>(val.ToString(), _opts)!;
+    }
+
+    public async ValueTask<bool> RemoveAsync<T>(TypedKey<T> key)
+    {
+        var db = _conn.GetDatabase();
+
+        return await db.KeyDeleteAsync(key.Key);
+    }
+
+    public async ValueTask<T?> GetOrAddAsync<T>(TypedKey<T> key, Func<Task<T?>> createFactory, TimeSpan? expiry = null)
+    {
+        var result = await GetAsync(key);
+
+        return await result.Match<Task<T?>>(
+            v => Task.FromResult<T?>(v),
+            async _ =>
+            {
+                var factoryValue = await createFactory();
+
+                if (factoryValue is null)
+                    return default;
+                
+                await AddAsync(key, factoryValue, expiry);
+                
+                // get again to make sure it's the cached value
+                // and not the late factory value, in case there's a race condition
+                
+                var newResult = await GetAsync(key);
+
+                // it's fine to do this, it should blow up if something went wrong.
+                return newResult.Match<T?>(
+                    v => v,
+                    _ => default);
+            });
+    }
+}

src/NadekoBot/Common/Collections/ConcurrentHashSet.cs

@@ -1,4 +1,4 @@
-#nullable disable
+#nullable enable
 #pragma warning disable
 // License MIT
 // Source: https://github.com/i3arnon/ConcurrentHashSet
@@ -7,433 +7,23 @@ using System.Diagnostics;
 
 namespace System.Collections.Generic;
 
-/// <summary>
-///     Represents a thread-safe hash-based unique collection.
-/// </summary>
-/// <typeparam name="T">The type of the items in the collection.</typeparam>
-/// <remarks>
-///     All public members of <see cref="ConcurrentHashSet{T}" /> are thread-safe and may be used
-///     concurrently from multiple threads.
-/// </remarks>
-[DebuggerDisplay("Count = {Count}")]
+[DebuggerDisplay("{_backingStore.Count}")]
 public sealed class ConcurrentHashSet<T> : IReadOnlyCollection<T>, ICollection<T>
 {
-    private const int DEFAULT_CAPACITY = 31;
-    private const int MAX_LOCK_NUMBER = 1024;
-
-    private static int DefaultConcurrencyLevel
-        => PlatformHelper.ProcessorCount;
-
-    /// <summary>
-    ///     Gets a value that indicates whether the <see cref="ConcurrentHashSet{T}" /> is empty.
-    /// </summary>
-    /// <value>
-    ///     true if the <see cref="ConcurrentHashSet{T}" /> is empty; otherwise,
-    ///     false.
-    /// </value>
-    public bool IsEmpty
-    {
-        get
-        {
-            var acquiredLocks = 0;
-            try
-            {
-                AcquireAllLocks(ref acquiredLocks);
-
-                for (var i = 0; i < tables.CountPerLock.Length; i++)
-                {
-                    if (tables.CountPerLock[i] != 0)
-                        return false;
-                }
-            }
-            finally
-            {
-                ReleaseLocks(0, acquiredLocks);
-            }
-
-            return true;
-        }
-    }
-
-    bool ICollection<T>.IsReadOnly
-        => false;
-
-    /// <summary>
-    ///     Gets the number of items contained in the
-    ///     <see
-    ///         cref="ConcurrentHashSet{T}" />
-    ///     .
-    /// </summary>
-    /// <value>
-    ///     The number of items contained in the
-    ///     <see
-    ///         cref="ConcurrentHashSet{T}" />
-    ///     .
-    /// </value>
-    /// <remarks>
-    ///     Count has snapshot semantics and represents the number of items in the
-    ///     <see
-    ///         cref="ConcurrentHashSet{T}" />
-    ///     at the moment when Count was accessed.
-    /// </remarks>
-    public int Count
-    {
-        get
-        {
-            var count = 0;
-            var acquiredLocks = 0;
-            try
-            {
-                AcquireAllLocks(ref acquiredLocks);
-
-                for (var i = 0; i < tables.CountPerLock.Length; i++)
-                    count += tables.CountPerLock[i];
-            }
-            finally
-            {
-                ReleaseLocks(0, acquiredLocks);
-            }
-
-            return count;
-        }
-    }
-
-    private readonly IEqualityComparer<T> _comparer;
-    private readonly bool _growLockArray;
-
-    private int budget;
-    private volatile Tables tables;
-
-    /// <summary>
-    ///     Initializes a new instance of the
-    ///     <see
-    ///         cref="ConcurrentHashSet{T}" />
-    ///     class that is empty, has the default concurrency level, has the default initial capacity, and
-    ///     uses the default comparer for the item type.
-    /// </summary>
+    private readonly ConcurrentDictionary<T, bool> _backingStore;
+    
     public ConcurrentHashSet()
-        : this(DefaultConcurrencyLevel, DEFAULT_CAPACITY, true, EqualityComparer<T>.Default)
-    {
-    }
+        => _backingStore = new();
 
-    /// <summary>
-    ///     Initializes a new instance of the
-    ///     <see
-    ///         cref="ConcurrentHashSet{T}" />
-    ///     class that is empty, has the specified concurrency level and capacity, and uses the default
-    ///     comparer for the item type.
-    /// </summary>
-    /// <param name="concurrencyLevel">
-    ///     The estimated number of threads that will update the
-    ///     <see cref="ConcurrentHashSet{T}" /> concurrently.
-    /// </param>
-    /// <param name="capacity">
-    ///     The initial number of elements that the
-    ///     <see
-    ///         cref="ConcurrentHashSet{T}" />
-    ///     can contain.
-    /// </param>
-    /// <exception cref="T:System.ArgumentOutOfRangeException">
-    ///     <paramref name="concurrencyLevel" /> is
-    ///     less than 1.
-    /// </exception>
-    /// <exception cref="T:System.ArgumentOutOfRangeException">
-    ///     <paramref name="capacity" /> is less than
-    ///     0.
-    /// </exception>
-    public ConcurrentHashSet(int concurrencyLevel, int capacity)
-        : this(concurrencyLevel, capacity, false, EqualityComparer<T>.Default)
-    {
-    }
+    public ConcurrentHashSet(IEnumerable<T> values, IEqualityComparer<T>? comparer = null)
+        => _backingStore = new(values.Select(x => new KeyValuePair<T, bool>(x, true)), comparer);
 
-    /// <summary>
-    ///     Initializes a new instance of the <see cref="ConcurrentHashSet{T}" />
-    ///     class that contains elements copied from the specified
-    ///     <see
-    ///         cref="T:System.Collections.IEnumerable{T}" />
-    ///     , has the default concurrency
-    ///     level, has the default initial capacity, and uses the default comparer for the item type.
-    /// </summary>
-    /// <param name="collection">
-    ///     The
-    ///     <see
-    ///         cref="T:System.Collections.IEnumerable{T}" />
-    ///     whose elements are copied to
-    ///     the new
-    ///     <see cref="ConcurrentHashSet{T}" />.
-    /// </param>
-    /// <exception cref="T:System.ArgumentNullException"><paramref name="collection" /> is a null reference.</exception>
-    public ConcurrentHashSet(IEnumerable<T> collection)
-        : this(collection, EqualityComparer<T>.Default)
-    {
-    }
-
-    /// <summary>
-    ///     Initializes a new instance of the <see cref="ConcurrentHashSet{T}" />
-    ///     class that is empty, has the specified concurrency level and capacity, and uses the specified
-    ///     <see cref="T:System.Collections.Generic.IEqualityComparer{T}" />.
-    /// </summary>
-    /// <param name="comparer">
-    ///     The <see cref="T:System.Collections.Generic.IEqualityComparer{T}" />
-    ///     implementation to use when comparing items.
-    /// </param>
-    /// <exception cref="T:System.ArgumentNullException"><paramref name="comparer" /> is a null reference.</exception>
-    public ConcurrentHashSet(IEqualityComparer<T> comparer)
-        : this(DefaultConcurrencyLevel, DEFAULT_CAPACITY, true, comparer)
-    {
-    }
-
-    /// <summary>
-    ///     Initializes a new instance of the <see cref="ConcurrentHashSet{T}" />
-    ///     class that contains elements copied from the specified
-    ///     <see
-    ///         cref="T:System.Collections.IEnumerable" />
-    ///     , has the default concurrency level, has the default
-    ///     initial capacity, and uses the specified
-    ///     <see cref="T:System.Collections.Generic.IEqualityComparer{T}" />.
-    /// </summary>
-    /// <param name="collection">
-    ///     The
-    ///     <see
-    ///         cref="T:System.Collections.IEnumerable{T}" />
-    ///     whose elements are copied to
-    ///     the new
-    ///     <see cref="ConcurrentHashSet{T}" />.
-    /// </param>
-    /// <param name="comparer">
-    ///     The <see cref="T:System.Collections.Generic.IEqualityComparer{T}" />
-    ///     implementation to use when comparing items.
-    /// </param>
-    /// <exception cref="T:System.ArgumentNullException">
-    ///     <paramref name="collection" /> is a null reference
-    ///     (Nothing in Visual Basic). -or-
-    ///     <paramref name="comparer" /> is a null reference (Nothing in Visual Basic).
-    /// </exception>
-    public ConcurrentHashSet(IEnumerable<T> collection, IEqualityComparer<T> comparer)
-        : this(comparer)
-    {
-        if (collection is null)
-            throw new ArgumentNullException(nameof(collection));
-
-        InitializeFromCollection(collection);
-    }
-
-
-    /// <summary>
-    ///     Initializes a new instance of the <see cref="ConcurrentHashSet{T}" />
-    ///     class that contains elements copied from the specified <see cref="T:System.Collections.IEnumerable" />,
-    ///     has the specified concurrency level, has the specified initial capacity, and uses the specified
-    ///     <see cref="T:System.Collections.Generic.IEqualityComparer{T}" />.
-    /// </summary>
-    /// <param name="concurrencyLevel">
-    ///     The estimated number of threads that will update the
-    ///     <see cref="ConcurrentHashSet{T}" /> concurrently.
-    /// </param>
-    /// <param name="collection">
-    ///     The <see cref="T:System.Collections.IEnumerable{T}" /> whose elements are copied to the new
-    ///     <see cref="ConcurrentHashSet{T}" />.
-    /// </param>
-    /// <param name="comparer">
-    ///     The <see cref="T:System.Collections.Generic.IEqualityComparer{T}" /> implementation to use
-    ///     when comparing items.
-    /// </param>
-    /// <exception cref="T:System.ArgumentNullException">
-    ///     <paramref name="collection" /> is a null reference.
-    ///     -or-
-    ///     <paramref name="comparer" /> is a null reference.
-    /// </exception>
-    /// <exception cref="T:System.ArgumentOutOfRangeException">
-    ///     <paramref name="concurrencyLevel" /> is less than 1.
-    /// </exception>
-    public ConcurrentHashSet(int concurrencyLevel, IEnumerable<T> collection, IEqualityComparer<T> comparer)
-        : this(concurrencyLevel, DEFAULT_CAPACITY, false, comparer)
-    {
-        if (collection is null)
-            throw new ArgumentNullException(nameof(collection));
-        if (comparer is null)
-            throw new ArgumentNullException(nameof(comparer));
-
-        InitializeFromCollection(collection);
-    }
-
-    /// <summary>
-    ///     Initializes a new instance of the <see cref="ConcurrentHashSet{T}" />
-    ///     class that is empty, has the specified concurrency level, has the specified initial capacity, and
-    ///     uses the specified <see cref="T:System.Collections.Generic.IEqualityComparer{T}" />.
-    /// </summary>
-    /// <param name="concurrencyLevel">
-    ///     The estimated number of threads that will update the
-    ///     <see cref="ConcurrentHashSet{T}" /> concurrently.
-    /// </param>
-    /// <param name="capacity">
-    ///     The initial number of elements that the
-    ///     <see
-    ///         cref="ConcurrentHashSet{T}" />
-    ///     can contain.
-    /// </param>
-    /// <param name="comparer">
-    ///     The <see cref="T:System.Collections.Generic.IEqualityComparer{T}" />
-    ///     implementation to use when comparing items.
-    /// </param>
-    /// <exception cref="T:System.ArgumentOutOfRangeException">
-    ///     <paramref name="concurrencyLevel" /> is less than 1. -or-
-    ///     <paramref name="capacity" /> is less than 0.
-    /// </exception>
-    /// <exception cref="T:System.ArgumentNullException"><paramref name="comparer" /> is a null reference.</exception>
-    public ConcurrentHashSet(int concurrencyLevel, int capacity, IEqualityComparer<T> comparer)
-        : this(concurrencyLevel, capacity, false, comparer)
-    {
-    }
-
-    private ConcurrentHashSet(
-        int concurrencyLevel,
-        int capacity,
-        bool growLockArray,
-        IEqualityComparer<T> comparer)
-    {
-        if (concurrencyLevel < 1)
-            throw new ArgumentOutOfRangeException(nameof(concurrencyLevel));
-        if (capacity < 0)
-            throw new ArgumentOutOfRangeException(nameof(capacity));
-
-        // The capacity should be at least as large as the concurrency level. Otherwise, we would have locks that don't guard
-        // any buckets.
-        if (capacity < concurrencyLevel)
-            capacity = concurrencyLevel;
-
-        var locks = new object[concurrencyLevel];
-        for (var i = 0; i < locks.Length; i++)
-            locks[i] = new();
-
-        var countPerLock = new int[locks.Length];
-        var buckets = new Node[capacity];
-        tables = new(buckets, locks, countPerLock);
-
-        _growLockArray = growLockArray;
-        budget = buckets.Length / locks.Length;
-        _comparer = comparer ?? throw new ArgumentNullException(nameof(comparer));
-    }
-
-    /// <summary>
-    ///     Removes all items from the <see cref="ConcurrentHashSet{T}" />.
-    /// </summary>
-    public void Clear()
-    {
-        var locksAcquired = 0;
-        try
-        {
-            AcquireAllLocks(ref locksAcquired);
-
-            var newTables = new Tables(new Node[DEFAULT_CAPACITY], tables.Locks, new int[tables.CountPerLock.Length]);
-            tables = newTables;
-            budget = Math.Max(1, newTables.Buckets.Length / newTables.Locks.Length);
-        }
-        finally
-        {
-            ReleaseLocks(0, locksAcquired);
-        }
-    }
-
-    /// <summary>
-    ///     Determines whether the <see cref="ConcurrentHashSet{T}" /> contains the specified
-    ///     item.
-    /// </summary>
-    /// <param name="item">The item to locate in the <see cref="ConcurrentHashSet{T}" />.</param>
-    /// <returns>true if the <see cref="ConcurrentHashSet{T}" /> contains the item; otherwise, false.</returns>
-    public bool Contains(T item)
-    {
-        var hashcode = _comparer.GetHashCode(item!);
-
-        // We must capture the _buckets field in a local variable. It is set to a new table on each table resize.
-        var localTables = tables;
-
-        var bucketNo = GetBucket(hashcode, localTables.Buckets.Length);
-
-        // We can get away w/out a lock here.
-        // The Volatile.Read ensures that the load of the fields of 'n' doesn't move before the load from buckets[i].
-        var current = Volatile.Read(ref localTables.Buckets[bucketNo]);
-
-        while (current is not null)
-        {
-            if (hashcode == current.Hashcode && _comparer.Equals(current.Item, item))
-                return true;
-
-            current = current.Next;
-        }
-
-        return false;
-    }
-
-    void ICollection<T>.Add(T item)
-        => Add(item);
-
-    void ICollection<T>.CopyTo(T[] array, int arrayIndex)
-    {
-        if (array is null)
-            throw new ArgumentNullException(nameof(array));
-        if (arrayIndex < 0)
-            throw new ArgumentOutOfRangeException(nameof(arrayIndex));
-
-        var locksAcquired = 0;
-        try
-        {
-            AcquireAllLocks(ref locksAcquired);
-
-            var count = 0;
-
-            for (var i = 0; i < tables.Locks.Length && count >= 0; i++)
-                count += tables.CountPerLock[i];
-
-            if (array.Length - count < arrayIndex || count < 0) //"count" itself or "count + arrayIndex" can overflow
-            {
-                throw new ArgumentException(
-                    "The index is equal to or greater than the length of the array, or the number of elements in the set is greater than the available space from index to the end of the destination array.");
-            }
-
-            CopyToItems(array, arrayIndex);
-        }
-        finally
-        {
-            ReleaseLocks(0, locksAcquired);
-        }
-    }
-
-    bool ICollection<T>.Remove(T item)
-        => TryRemove(item);
+    public IEnumerator<T> GetEnumerator()
+        => _backingStore.Keys.GetEnumerator();
 
     IEnumerator IEnumerable.GetEnumerator()
         => GetEnumerator();
 
-    /// <summary>
-    ///     Returns an enumerator that iterates through the
-    ///     <see
-    ///         cref="ConcurrentHashSet{T}" />
-    ///     .
-    /// </summary>
-    /// <returns>An enumerator for the <see cref="ConcurrentHashSet{T}" />.</returns>
-    /// <remarks>
-    ///     The enumerator returned from the collection is safe to use concurrently with
-    ///     reads and writes to the collection, however it does not represent a moment-in-time snapshot
-    ///     of the collection.  The contents exposed through the enumerator may contain modifications
-    ///     made to the collection after <see cref="GetEnumerator" /> was called.
-    /// </remarks>
-    public IEnumerator<T> GetEnumerator()
-    {
-        var buckets = tables.Buckets;
-
-        for (var i = 0; i < buckets.Length; i++)
-        {
-            // The Volatile.Read ensures that the load of the fields of 'current' doesn't move before the load from buckets[i].
-            var current = Volatile.Read(ref buckets[i]);
-
-            while (current is not null)
-            {
-                yield return current.Item;
-                current = current.Next;
-            }
-        }
-    }
-
     /// <summary>
     ///     Adds the specified item to the <see cref="ConcurrentHashSet{T}" />.
     /// </summary>
@@ -447,375 +37,57 @@ public sealed class ConcurrentHashSet<T> : IReadOnlyCollection<T>, ICollection<T
     ///     contains too many items.
     /// </exception>
     public bool Add(T item)
-        => AddInternal(item, _comparer.GetHashCode(item), true);
+        => _backingStore.TryAdd(item, true);
+
+    void ICollection<T>.Add(T item)
+        => Add(item);
+
+    public void Clear()
+        => _backingStore.Clear();
+
+    public bool Contains(T item)
+        => _backingStore.ContainsKey(item);
+
+    public void CopyTo(T[] array, int arrayIndex)
+    {
+        ArgumentNullException.ThrowIfNull(array);
+        
+        if (arrayIndex < 0)
+            throw new ArgumentOutOfRangeException(nameof(arrayIndex));
+        
+        if (arrayIndex >= array.Length)
+            throw new ArgumentOutOfRangeException(nameof(arrayIndex));
+
+        CopyToInternal(array, arrayIndex);
+    }
+
+    private void CopyToInternal(T[] array, int arrayIndex)
+    {
+        var len = array.Length;
+        foreach (var (k, _) in _backingStore)
+        {
+            if (arrayIndex >= len)
+                throw new IndexOutOfRangeException(nameof(arrayIndex));
+            
+            array[arrayIndex++] = k;
+        }
+    }
+
+    bool ICollection<T>.Remove(T item)
+        => TryRemove(item);
 
-    /// <summary>
-    ///     Attempts to remove the item from the <see cref="ConcurrentHashSet{T}" />.
-    /// </summary>
-    /// <param name="item">The item to remove.</param>
-    /// <returns>true if an item was removed successfully; otherwise, false.</returns>
     public bool TryRemove(T item)
+        => _backingStore.TryRemove(item, out _);
+
+    public void RemoveWhere(Func<T, bool> predicate)
     {
-        var hashcode = _comparer.GetHashCode(item);
-        while (true)
-        {
-            var localTables = tables;
-
-            GetBucketAndLockNo(hashcode,
-                out var bucketNo,
-                out var lockNo,
-                localTables.Buckets.Length,
-                localTables.Locks.Length);
-
-            lock (localTables.Locks[lockNo])
-            {
-                // If the table just got resized, we may not be holding the right lock, and must retry.
-                // This should be a rare occurrence.
-                if (localTables != tables)
-                    continue;
-
-                Node previous = null;
-                for (var current = localTables.Buckets[bucketNo]; current is not null; current = current.Next)
-                {
-                    Debug.Assert((previous is null && current == localTables.Buckets[bucketNo])
-                                 || previous!.Next == current);
-
-                    if (hashcode == current.Hashcode && _comparer.Equals(current.Item, item))
-                    {
-                        if (previous is null)
-                            Volatile.Write(ref localTables.Buckets[bucketNo], current.Next);
-                        else
-                            previous.Next = current.Next;
-
-                        localTables.CountPerLock[lockNo]--;
-                        return true;
-                    }
-
-                    previous = current;
-                }
-            }
-
-            return false;
-        }
+        foreach (var elem in this.Where(predicate))
+            TryRemove(elem);
     }
 
-    private void InitializeFromCollection(IEnumerable<T> collection)
-    {
-        foreach (var item in collection)
-            AddInternal(item, _comparer.GetHashCode(item), false);
+    public int Count
+        => _backingStore.Count;
 
-        if (budget == 0)
-            budget = tables.Buckets.Length / tables.Locks.Length;
-    }
-
-    private bool AddInternal(T item, int hashcode, bool acquireLock)
-    {
-        while (true)
-        {
-            var localTables = tables;
-            GetBucketAndLockNo(hashcode,
-                out var bucketNo,
-                out var lockNo,
-                localTables.Buckets.Length,
-                localTables.Locks.Length);
-
-            var resizeDesired = false;
-            var lockTaken = false;
-            try
-            {
-                if (acquireLock)
-                    Monitor.Enter(localTables.Locks[lockNo], ref lockTaken);
-
-                // If the table just got resized, we may not be holding the right lock, and must retry.
-                // This should be a rare occurrence.
-                if (localTables != tables)
-                    continue;
-
-                // Try to find this item in the bucket
-                Node previous = null;
-                for (var current = localTables.Buckets[bucketNo]; current is not null; current = current.Next)
-                {
-                    Debug.Assert((previous is null && current == localTables.Buckets[bucketNo])
-                                 || previous!.Next == current);
-                    if (hashcode == current.Hashcode && _comparer.Equals(current.Item, item))
-                        return false;
-
-                    previous = current;
-                }
-
-                // The item was not found in the bucket. Insert the new item.
-                Volatile.Write(ref localTables.Buckets[bucketNo], new(item, hashcode, localTables.Buckets[bucketNo]));
-                checked
-                {
-                    localTables.CountPerLock[lockNo]++;
-                }
-
-                //
-                // If the number of elements guarded by this lock has exceeded the budget, resize the bucket table.
-                // It is also possible that GrowTable will increase the budget but won't resize the bucket table.
-                // That happens if the bucket table is found to be poorly utilized due to a bad hash function.
-                //
-                if (localTables.CountPerLock[lockNo] > budget)
-                    resizeDesired = true;
-            }
-            finally
-            {
-                if (lockTaken)
-                    Monitor.Exit(localTables.Locks[lockNo]);
-            }
-
-            //
-            // The fact that we got here means that we just performed an insertion. If necessary, we will grow the table.
-            //
-            // Concurrency notes:
-            // - Notice that we are not holding any locks at when calling GrowTable. This is necessary to prevent deadlocks.
-            // - As a result, it is possible that GrowTable will be called unnecessarily. But, GrowTable will obtain lock 0
-            //   and then verify that the table we passed to it as the argument is still the current table.
-            //
-            if (resizeDesired)
-                GrowTable(localTables);
-
-            return true;
-        }
-    }
-
-    private static int GetBucket(int hashcode, int bucketCount)
-    {
-        var bucketNo = (hashcode & 0x7fffffff) % bucketCount;
-        Debug.Assert(bucketNo >= 0 && bucketNo < bucketCount);
-        return bucketNo;
-    }
-
-    private static void GetBucketAndLockNo(
-        int hashcode,
-        out int bucketNo,
-        out int lockNo,
-        int bucketCount,
-        int lockCount)
-    {
-        bucketNo = (hashcode & 0x7fffffff) % bucketCount;
-        lockNo = bucketNo % lockCount;
-
-        Debug.Assert(bucketNo >= 0 && bucketNo < bucketCount);
-        Debug.Assert(lockNo >= 0 && lockNo < lockCount);
-    }
-
-    private void GrowTable(Tables localTables)
-    {
-        const int maxArrayLength = 0X7FEFFFFF;
-        var locksAcquired = 0;
-        try
-        {
-            // The thread that first obtains _locks[0] will be the one doing the resize operation
-            AcquireLocks(0, 1, ref locksAcquired);
-
-            // Make sure nobody resized the table while we were waiting for lock 0:
-            if (localTables != tables)
-                // We assume that since the table reference is different, it was already resized (or the budget
-                // was adjusted). If we ever decide to do table shrinking, or replace the table for other reasons,
-                // we will have to revisit this logic.
-                return;
-
-            // Compute the (approx.) total size. Use an Int64 accumulation variable to avoid an overflow.
-            long approxCount = 0;
-            for (var i = 0; i < localTables.CountPerLock.Length; i++)
-                approxCount += localTables.CountPerLock[i];
-
-            //
-            // If the bucket array is too empty, double the budget instead of resizing the table
-            //
-            if (approxCount < localTables.Buckets.Length / 4)
-            {
-                budget = 2 * budget;
-                if (budget < 0)
-                    budget = int.MaxValue;
-
-                return;
-            }
-
-            // Compute the new table size. We find the smallest integer larger than twice the previous table size, and not divisible by
-            // 2,3,5 or 7. We can consider a different table-sizing policy in the future.
-            var newLength = 0;
-            var maximizeTableSize = false;
-            try
-            {
-                checked
-                {
-                    // Double the size of the buckets table and add one, so that we have an odd integer.
-                    newLength = (localTables.Buckets.Length * 2) + 1;
-
-                    // Now, we only need to check odd integers, and find the first that is not divisible
-                    // by 3, 5 or 7.
-                    while (newLength % 3 == 0 || newLength % 5 == 0 || newLength % 7 == 0)
-                        newLength += 2;
-
-                    Debug.Assert(newLength % 2 != 0);
-
-                    if (newLength > maxArrayLength)
-                        maximizeTableSize = true;
-                }
-            }
-            catch (OverflowException)
-            {
-                maximizeTableSize = true;
-            }
-
-            if (maximizeTableSize)
-            {
-                newLength = maxArrayLength;
-
-                // We want to make sure that GrowTable will not be called again, since table is at the maximum size.
-                // To achieve that, we set the budget to int.MaxValue.
-                //
-                // (There is one special case that would allow GrowTable() to be called in the future: 
-                // calling Clear() on the ConcurrentHashSet will shrink the table and lower the budget.)
-                budget = int.MaxValue;
-            }
-
-            // Now acquire all other locks for the table
-            AcquireLocks(1, localTables.Locks.Length, ref locksAcquired);
-
-            var newLocks = localTables.Locks;
-
-            // Add more locks
-            if (_growLockArray && localTables.Locks.Length < MAX_LOCK_NUMBER)
-            {
-                newLocks = new object[localTables.Locks.Length * 2];
-                Array.Copy(localTables.Locks, 0, newLocks, 0, localTables.Locks.Length);
-                for (var i = localTables.Locks.Length; i < newLocks.Length; i++)
-                    newLocks[i] = new();
-            }
-
-            var newBuckets = new Node[newLength];
-            var newCountPerLock = new int[newLocks.Length];
-
-            // Copy all data into a new table, creating new nodes for all elements
-            for (var i = 0; i < localTables.Buckets.Length; i++)
-            {
-                var current = localTables.Buckets[i];
-                while (current is not null)
-                {
-                    var next = current.Next;
-                    GetBucketAndLockNo(current.Hashcode,
-                        out var newBucketNo,
-                        out var newLockNo,
-                        newBuckets.Length,
-                        newLocks.Length);
-
-                    newBuckets[newBucketNo] = new(current.Item, current.Hashcode, newBuckets[newBucketNo]);
-
-                    checked
-                    {
-                        newCountPerLock[newLockNo]++;
-                    }
-
-                    current = next;
-                }
-            }
-
-            // Adjust the budget
-            budget = Math.Max(1, newBuckets.Length / newLocks.Length);
-
-            // Replace tables with the new versions
-            tables = new(newBuckets, newLocks, newCountPerLock);
-        }
-        finally
-        {
-            // Release all locks that we took earlier
-            ReleaseLocks(0, locksAcquired);
-        }
-    }
-
-    public int RemoveWhere(Func<T, bool> predicate)
-    {
-        var elems = this.Where(predicate);
-        var removed = 0;
-        foreach (var elem in elems)
-        {
-            if (TryRemove(elem))
-                removed++;
-        }
-
-        return removed;
-    }
-
-    private void AcquireAllLocks(ref int locksAcquired)
-    {
-        // First, acquire lock 0
-        AcquireLocks(0, 1, ref locksAcquired);
-
-        // Now that we have lock 0, the _locks array will not change (i.e., grow),
-        // and so we can safely read _locks.Length.
-        AcquireLocks(1, tables.Locks.Length, ref locksAcquired);
-        Debug.Assert(locksAcquired == tables.Locks.Length);
-    }
-
-    private void AcquireLocks(int fromInclusive, int toExclusive, ref int locksAcquired)
-    {
-        Debug.Assert(fromInclusive <= toExclusive);
-        var locks = tables.Locks;
-
-        for (var i = fromInclusive; i < toExclusive; i++)
-        {
-            var lockTaken = false;
-            try
-            {
-                Monitor.Enter(locks[i], ref lockTaken);
-            }
-            finally
-            {
-                if (lockTaken)
-                    locksAcquired++;
-            }
-        }
-    }
-
-    private void ReleaseLocks(int fromInclusive, int toExclusive)
-    {
-        Debug.Assert(fromInclusive <= toExclusive);
-
-        for (var i = fromInclusive; i < toExclusive; i++)
-            Monitor.Exit(tables.Locks[i]);
-    }
-
-    private void CopyToItems(T[] array, int index)
-    {
-        var buckets = tables.Buckets;
-        for (var i = 0; i < buckets.Length; i++)
-        for (var current = buckets[i]; current is not null; current = current.Next)
-        {
-            array[index] = current.Item;
-            index++; //this should never flow, CopyToItems is only called when there's no overflow risk
-        }
-    }
-
-    private sealed class Tables
-    {
-        public readonly Node[] Buckets;
-        public readonly object[] Locks;
-
-        public volatile int[] CountPerLock;
-
-        public Tables(Node[] buckets, object[] locks, int[] countPerLock)
-        {
-            Buckets = buckets;
-            Locks = locks;
-            CountPerLock = countPerLock;
-        }
-    }
-
-    private sealed class Node
-    {
-        public readonly int Hashcode;
-        public readonly T Item;
-
-        public volatile Node Next;
-
-        public Node(T item, int hashcode, Node next)
-        {
-            Item = item;
-            Hashcode = hashcode;
-            Next = next;
-        }
-    }
+    public bool IsReadOnly
+        => false;
 }

src/NadekoBot/Common/Configs/BotConfig.cs

@@ -12,7 +12,7 @@ namespace NadekoBot.Common.Configs;
 public sealed partial class BotConfig : ICloneable<BotConfig>
 {
     [Comment(@"DO NOT CHANGE")]
-    public int Version { get; set; } = 2;
+    public int Version { get; set; } = 3;
 
     [Comment(@"Most commands, when executed, have a small colored line
 next to the response. The color depends whether the command
@@ -182,4 +182,4 @@ public enum ConsoleOutputType
     Normal = 0,
     Simple = 1,
     None = 2
-}
+}

src/NadekoBot/Common/Creds.cs

@@ -19,7 +19,10 @@ public sealed class Creds : IBotCredentials
     public bool UsePrivilegedIntents { get; set; }
 
     [Comment(@"The number of shards that the bot will be running on.
-Leave at 1 if you don't know what you're doing.")]
+Leave at 1 if you don't know what you're doing.
+
+note: If you are planning to have more than one shard, then you must change botCache to 'redis'.
+      Also, in that case you should be using NadekoBot.Coordinator to start the bot, and it will correctly override this value.")]
     public int TotalShards { get; set; }
 
     [Comment(   
@@ -50,8 +53,14 @@ go to https://www.patreon.com/portal -> my clients -> create client")]
 
     [Comment(@"Official cleverbot api key.")]
     public string CleverbotApiKey { get; set; }
-
-    [Comment(@"Redis connection string. Don't change if you don't know what you're doing.")]
+    
+    [Comment(@"Which cache implementation should bot use.
+'memory' - Cache will be in memory of the bot's process itself. Only use this on bots with a single shard. When the bot is restarted the cache is reset. 
+'redis' - Uses redis (which needs to be separately downloaded and installed). The cache will persist through bot restarts. You can configure connection string in creds.yml")]
+    public BotCacheImplemenation BotCache { get; set; }
+    
+    [Comment(@"Redis connection string. Don't change if you don't know what you're doing.
+Only used if botCache is set to 'redis'")]
     public string RedisOptions { get; set; }
 
     [Comment(@"Database options. Don't change if you don't know what you're doing. Leave null for default values")]
@@ -104,12 +113,12 @@ Linux default
     args: ""NadekoBot.dll -- {0}""
 Windows default
     cmd: NadekoBot.exe
-    args: {0}")]
+    args: ""{0}""")]
     public RestartConfig RestartCommand { get; set; }
 
     public Creds()
     {
-        Version = 5;
+        Version = 6;
         Token = string.Empty;
         UsePrivilegedIntents = true;
         OwnerIds = new List<ulong>();
@@ -119,6 +128,7 @@ Windows default
         Patreon = new(string.Empty, string.Empty, string.Empty, string.Empty);
         BotListToken = string.Empty;
         CleverbotApiKey = string.Empty;
+        BotCache = BotCacheImplemenation.Memory;
         RedisOptions = "localhost:6379,syncTimeout=30000,responseTimeout=30000,allowAdmin=true,password=";
         Db = new()
         {
@@ -217,4 +227,10 @@ public class GoogleApiConfig
 {
     public string SearchId { get; init; }
     public string ImageSearchId { get; init; }
+}
+
+public enum BotCacheImplemenation
+{
+    Memory,
+    Redis
 }

src/NadekoBot/Common/IBotCredentials.cs

@@ -26,6 +26,7 @@ public interface IBotCredentials
     string TwitchClientId { get; set; }
     string TwitchClientSecret { get; set; }
     GoogleApiConfig Google { get; set; }
+    BotCacheImplemenation BotCache { get; set; }
 }
 
 public class RestartConfig

src/NadekoBot/Common/ImageUrls.cs

@@ -1,9 +1,11 @@
 #nullable disable
 using NadekoBot.Common.Yml;
+using Cloneable;
 
 namespace NadekoBot.Common;
 
-public class ImageUrls
+[Cloneable]
+public partial class ImageUrls : ICloneable<ImageUrls> 
 {
     [Comment("DO NOT CHANGE")]
     public int Version { get; set; } = 3;

src/NadekoBot/Common/OneOfExtensions.cs

@@ -0,0 +1,10 @@
+using OneOf.Types;
+using OneOf;
+
+namespace NadekoBot.Common;
+
+public static class OneOfExtensions
+{
+    public static bool TryGetValue<T>(this OneOf<T, None> oneOf, out T value)
+        => oneOf.TryPickT0(out value, out _);
+}

src/NadekoBot/Common/Pokemon/SearchPokemon.cs

@@ -1,11 +1,12 @@
 #nullable disable
-using Newtonsoft.Json;
+
+using System.Text.Json.Serialization;
 
 namespace NadekoBot.Common.Pokemon;
 
 public class SearchPokemon
 {
-    [JsonProperty("num")]
+    [JsonPropertyName("num")]
     public int Id { get; set; }
 
     public string Species { get; set; }

src/NadekoBot/Common/QueueRunner.cs

@@ -0,0 +1,62 @@
+using System.Threading.Channels;
+
+namespace NadekoBot.Common;
+
+public sealed class QueueRunner
+{
+    private readonly Channel<Func<Task>> _channel;
+    private readonly int _delayMs;
+
+    public QueueRunner(int delayMs = 0, int maxCapacity = -1)
+    {
+        if (delayMs < 0)
+            throw new ArgumentOutOfRangeException(nameof(delayMs));
+
+        _delayMs = delayMs;
+        _channel = maxCapacity switch
+        {
+            0 or < -1 => throw new ArgumentOutOfRangeException(nameof(maxCapacity)),
+            -1 => Channel.CreateUnbounded<Func<Task>>(new UnboundedChannelOptions()
+            {
+                SingleReader = true,
+                SingleWriter = false,
+                AllowSynchronousContinuations = true,
+            }),
+            _ => Channel.CreateBounded<Func<Task>>(new BoundedChannelOptions(maxCapacity)
+            {
+                Capacity = maxCapacity,
+                FullMode = BoundedChannelFullMode.DropOldest,
+                SingleReader = true,
+                SingleWriter = false,
+                AllowSynchronousContinuations = true
+            })
+        };
+    }
+
+    public async Task RunAsync(CancellationToken cancel = default)
+    {
+        while (true)
+        {
+            var func = await _channel.Reader.ReadAsync(cancel);
+            
+            try
+            {
+                await func();
+            }
+            catch (Exception ex)
+            {
+                Log.Warning(ex, "Exception executing a staggered func: {ErrorMessage}", ex.Message);
+            }
+            finally
+            {
+                if (_delayMs != 0)
+                {
+                    await Task.Delay(_delayMs, cancel);
+                }
+            }
+        }
+    }
+    
+    public ValueTask Enqueue(Func<Task> action)
+        => _channel.Writer.WriteAsync(action);
+}

src/NadekoBot/Common/TypeReaders/BotCommandTypeReader.cs

@@ -31,38 +31,38 @@ public sealed class CommandTypeReader : NadekoTypeReader<CommandInfo>
     }
 }
 
-public sealed class CommandOrCrTypeReader : NadekoTypeReader<CommandOrCrInfo>
+public sealed class CommandOrExprTypeReader : NadekoTypeReader<CommandOrExprInfo>
 {
     private readonly CommandService _cmds;
     private readonly CommandHandler _commandHandler;
     private readonly NadekoExpressionsService _exprs;
 
-    public CommandOrCrTypeReader(CommandService cmds, NadekoExpressionsService exprs, CommandHandler commandHandler)
+    public CommandOrExprTypeReader(CommandService cmds, NadekoExpressionsService exprs, CommandHandler commandHandler)
     {
         _cmds = cmds;
         _exprs = exprs;
         _commandHandler = commandHandler;
     }
 
-    public override async ValueTask<TypeReaderResult<CommandOrCrInfo>> ReadAsync(ICommandContext ctx, string input)
+    public override async ValueTask<TypeReaderResult<CommandOrExprInfo>> ReadAsync(ICommandContext ctx, string input)
     {
         input = input.ToUpperInvariant();
 
-        if (_exprs.ExpressionExists(ctx.Guild?.Id, input))
-            return TypeReaderResult.FromSuccess(new CommandOrCrInfo(input, CommandOrCrInfo.Type.Custom));
+        if (_exprs.ExpressionExists(ctx.Guild?.Id, input) || _exprs.ExpressionExists(null, input))
+            return TypeReaderResult.FromSuccess(new CommandOrExprInfo(input, CommandOrExprInfo.Type.Custom));
 
         var cmd = await new CommandTypeReader(_commandHandler, _cmds).ReadAsync(ctx, input);
         if (cmd.IsSuccess)
         {
-            return TypeReaderResult.FromSuccess(new CommandOrCrInfo(((CommandInfo)cmd.Values.First().Value).Name,
-                CommandOrCrInfo.Type.Normal));
+            return TypeReaderResult.FromSuccess(new CommandOrExprInfo(((CommandInfo)cmd.Values.First().Value).Name,
+                CommandOrExprInfo.Type.Normal));
         }
 
-        return TypeReaderResult.FromError<CommandOrCrInfo>(CommandError.ParseFailed, "No such command or cr found.");
+        return TypeReaderResult.FromError<CommandOrExprInfo>(CommandError.ParseFailed, "No such command or expression found.");
     }
 }
 
-public class CommandOrCrInfo
+public class CommandOrExprInfo
 {
     public enum Type
     {
@@ -76,7 +76,7 @@ public class CommandOrCrInfo
     public bool IsCustom
         => CmdType == Type.Custom;
 
-    public CommandOrCrInfo(string input, Type type)
+    public CommandOrExprInfo(string input, Type type)
     {
         Name = input;
         CmdType = type;

src/NadekoBot/Db/Models/PatronQuota.cs

@@ -35,4 +35,14 @@ public class PatronUser
     
     // Date Only component
     public DateTime ValidThru { get; set; }
+    
+    public PatronUser Clone()
+        => new PatronUser()
+        {
+            UniquePlatformUserId = this.UniquePlatformUserId,
+            UserId = this.UserId,
+            AmountCents = this.AmountCents,
+            LastCharge = this.LastCharge,
+            ValidThru = this.ValidThru
+        };
 }

src/NadekoBot/Db/Models/StreamOnlineMessage.cs

@@ -0,0 +1,13 @@
+#nullable disable
+using NadekoBot.Services.Database.Models;
+
+namespace NadekoBot.Db.Models;
+
+public class StreamOnlineMessage : DbEntity
+{
+    public ulong ChannelId { get; set; }
+    public ulong MessageId { get; set; }
+
+    public FollowedStream.FType Type { get; set; }
+    public string Name { get; set; }
+}

src/NadekoBot/Db/NadekoContext.cs

@@ -58,6 +58,9 @@ public abstract class NadekoContext : DbContext
     public DbSet<PatronUser> Patrons { get; set; }
 
     public DbSet<PatronQuota> PatronQuotas { get; set; }
+    
+    public DbSet<StreamOnlineMessage> StreamOnlineMessages { get; set; }
+
 
     #region Mandatory Provider-Specific Values
 

src/NadekoBot/GlobalUsings.cs

@@ -1,4 +1,5 @@
-global using System.Collections.Concurrent;
+// global using System.Collections.Concurrent;
+global using NonBlocking;
 
 // packages
 global using Serilog;

src/NadekoBot/Migrations/MySql/20220623090718_stondel-db-cache.cs

@@ -0,0 +1,39 @@
+using System;
+using Microsoft.EntityFrameworkCore.Metadata;
+using Microsoft.EntityFrameworkCore.Migrations;
+
+#nullable disable
+
+namespace NadekoBot.Migrations.Mysql
+{
+    public partial class stondeldbcache : Migration
+    {
+        protected override void Up(MigrationBuilder migrationBuilder)
+        {
+            migrationBuilder.CreateTable(
+                name: "streamonlinemessages",
+                columns: table => new
+                {
+                    id = table.Column<int>(type: "int", nullable: false)
+                        .Annotation("MySql:ValueGenerationStrategy", MySqlValueGenerationStrategy.IdentityColumn),
+                    channelid = table.Column<ulong>(type: "bigint unsigned", nullable: false),
+                    messageid = table.Column<ulong>(type: "bigint unsigned", nullable: false),
+                    type = table.Column<int>(type: "int", nullable: false),
+                    name = table.Column<string>(type: "longtext", nullable: true)
+                        .Annotation("MySql:CharSet", "utf8mb4"),
+                    dateadded = table.Column<DateTime>(type: "datetime(6)", nullable: true)
+                },
+                constraints: table =>
+                {
+                    table.PrimaryKey("pk_streamonlinemessages", x => x.id);
+                })
+                .Annotation("MySql:CharSet", "utf8mb4");
+        }
+
+        protected override void Down(MigrationBuilder migrationBuilder)
+        {
+            migrationBuilder.DropTable(
+                name: "streamonlinemessages");
+        }
+    }
+}

src/NadekoBot/Migrations/MySql/MysqlContextModelSnapshot.cs

@@ -16,7 +16,7 @@ namespace NadekoBot.Migrations.Mysql
         {
 #pragma warning disable 612, 618
             modelBuilder
-                .HasAnnotation("ProductVersion", "6.0.5")
+                .HasAnnotation("ProductVersion", "6.0.6")
                 .HasAnnotation("Relational:MaxIdentifierLength", 64);
 
             modelBuilder.Entity("NadekoBot.Db.Models.BankUser", b =>
@@ -333,6 +333,39 @@ namespace NadekoBot.Migrations.Mysql
                     b.ToTable("patrons", (string)null);
                 });
 
+            modelBuilder.Entity("NadekoBot.Db.Models.StreamOnlineMessage", b =>
+                {
+                    b.Property<int>("Id")
+                        .ValueGeneratedOnAdd()
+                        .HasColumnType("int")
+                        .HasColumnName("id");
+
+                    b.Property<ulong>("ChannelId")
+                        .HasColumnType("bigint unsigned")
+                        .HasColumnName("channelid");
+
+                    b.Property<DateTime?>("DateAdded")
+                        .HasColumnType("datetime(6)")
+                        .HasColumnName("dateadded");
+
+                    b.Property<ulong>("MessageId")
+                        .HasColumnType("bigint unsigned")
+                        .HasColumnName("messageid");
+
+                    b.Property<string>("Name")
+                        .HasColumnType("longtext")
+                        .HasColumnName("name");
+
+                    b.Property<int>("Type")
+                        .HasColumnType("int")
+                        .HasColumnName("type");
+
+                    b.HasKey("Id")
+                        .HasName("pk_streamonlinemessages");
+
+                    b.ToTable("streamonlinemessages", (string)null);
+                });
+
             modelBuilder.Entity("NadekoBot.Services.Database.Models.AntiAltSetting", b =>
                 {
                     b.Property<int>("Id")

src/NadekoBot/Migrations/Postgresql/20220623090729_stondel-db-cache.cs

@@ -0,0 +1,37 @@
+using System;
+using Microsoft.EntityFrameworkCore.Migrations;
+using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata;
+
+#nullable disable
+
+namespace NadekoBot.Migrations.PostgreSql
+{
+    public partial class stondeldbcache : Migration
+    {
+        protected override void Up(MigrationBuilder migrationBuilder)
+        {
+            migrationBuilder.CreateTable(
+                name: "streamonlinemessages",
+                columns: table => new
+                {
+                    id = table.Column<int>(type: "integer", nullable: false)
+                        .Annotation("Npgsql:ValueGenerationStrategy", NpgsqlValueGenerationStrategy.IdentityByDefaultColumn),
+                    channelid = table.Column<decimal>(type: "numeric(20,0)", nullable: false),
+                    messageid = table.Column<decimal>(type: "numeric(20,0)", nullable: false),
+                    type = table.Column<int>(type: "integer", nullable: false),
+                    name = table.Column<string>(type: "text", nullable: true),
+                    dateadded = table.Column<DateTime>(type: "timestamp with time zone", nullable: true)
+                },
+                constraints: table =>
+                {
+                    table.PrimaryKey("pk_streamonlinemessages", x => x.id);
+                });
+        }
+
+        protected override void Down(MigrationBuilder migrationBuilder)
+        {
+            migrationBuilder.DropTable(
+                name: "streamonlinemessages");
+        }
+    }
+}

src/NadekoBot/Migrations/Postgresql/PostgreSqlContextModelSnapshot.cs

@@ -17,7 +17,7 @@ namespace NadekoBot.Migrations.PostgreSql
         {
 #pragma warning disable 612, 618
             modelBuilder
-                .HasAnnotation("ProductVersion", "6.0.5")
+                .HasAnnotation("ProductVersion", "6.0.6")
                 .HasAnnotation("Relational:MaxIdentifierLength", 63);
 
             NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder);
@@ -343,6 +343,41 @@ namespace NadekoBot.Migrations.PostgreSql
                     b.ToTable("patrons", (string)null);
                 });
 
+            modelBuilder.Entity("NadekoBot.Db.Models.StreamOnlineMessage", b =>
+                {
+                    b.Property<int>("Id")
+                        .ValueGeneratedOnAdd()
+                        .HasColumnType("integer")
+                        .HasColumnName("id");
+
+                    NpgsqlPropertyBuilderExtensions.UseIdentityByDefaultColumn(b.Property<int>("Id"));
+
+                    b.Property<decimal>("ChannelId")
+                        .HasColumnType("numeric(20,0)")
+                        .HasColumnName("channelid");
+
+                    b.Property<DateTime?>("DateAdded")
+                        .HasColumnType("timestamp with time zone")
+                        .HasColumnName("dateadded");
+
+                    b.Property<decimal>("MessageId")
+                        .HasColumnType("numeric(20,0)")
+                        .HasColumnName("messageid");
+
+                    b.Property<string>("Name")
+                        .HasColumnType("text")
+                        .HasColumnName("name");
+
+                    b.Property<int>("Type")
+                        .HasColumnType("integer")
+                        .HasColumnName("type");
+
+                    b.HasKey("Id")
+                        .HasName("pk_streamonlinemessages");
+
+                    b.ToTable("streamonlinemessages", (string)null);
+                });
+
             modelBuilder.Entity("NadekoBot.Services.Database.Models.AntiAltSetting", b =>
                 {
                     b.Property<int>("Id")

src/NadekoBot/Migrations/Sqlite/20220623073903_stondel-db-cache.cs

@@ -0,0 +1,36 @@
+using System;
+using Microsoft.EntityFrameworkCore.Migrations;
+
+#nullable disable
+
+namespace NadekoBot.Migrations
+{
+    public partial class stondeldbcache : Migration
+    {
+        protected override void Up(MigrationBuilder migrationBuilder)
+        {
+            migrationBuilder.CreateTable(
+                name: "StreamOnlineMessages",
+                columns: table => new
+                {
+                    Id = table.Column<int>(type: "INTEGER", nullable: false)
+                        .Annotation("Sqlite:Autoincrement", true),
+                    ChannelId = table.Column<ulong>(type: "INTEGER", nullable: false),
+                    MessageId = table.Column<ulong>(type: "INTEGER", nullable: false),
+                    Type = table.Column<int>(type: "INTEGER", nullable: false),
+                    Name = table.Column<string>(type: "TEXT", nullable: false),
+                    DateAdded = table.Column<DateTime>(type: "TEXT", nullable: true)
+                },
+                constraints: table =>
+                {
+                    table.PrimaryKey("PK_StreamOnlineMessages", x => x.Id);
+                });
+        }
+
+        protected override void Down(MigrationBuilder migrationBuilder)
+        {
+            migrationBuilder.DropTable(
+                name: "StreamOnlineMessages");
+        }
+    }
+}

src/NadekoBot/Migrations/Sqlite/NadekoSqliteContextModelSnapshot.cs

@@ -15,7 +15,7 @@ namespace NadekoBot.Migrations
         protected override void BuildModel(ModelBuilder modelBuilder)
         {
 #pragma warning disable 612, 618
-            modelBuilder.HasAnnotation("ProductVersion", "6.0.5");
+            modelBuilder.HasAnnotation("ProductVersion", "6.0.6");
 
             modelBuilder.Entity("NadekoBot.Db.Models.BankUser", b =>
                 {
@@ -262,6 +262,33 @@ namespace NadekoBot.Migrations
                     b.ToTable("Patrons");
                 });
 
+            modelBuilder.Entity("NadekoBot.Db.Models.StreamOnlineMessage", b =>
+                {
+                    b.Property<int>("Id")
+                        .ValueGeneratedOnAdd()
+                        .HasColumnType("INTEGER");
+
+                    b.Property<ulong>("ChannelId")
+                        .HasColumnType("INTEGER");
+
+                    b.Property<DateTime?>("DateAdded")
+                        .HasColumnType("TEXT");
+
+                    b.Property<ulong>("MessageId")
+                        .HasColumnType("INTEGER");
+
+                    b.Property<string>("Name")
+                        .IsRequired()
+                        .HasColumnType("TEXT");
+
+                    b.Property<int>("Type")
+                        .HasColumnType("INTEGER");
+
+                    b.HasKey("Id");
+
+                    b.ToTable("StreamOnlineMessages");
+                });
+
             modelBuilder.Entity("NadekoBot.Services.Database.Models.AntiAltSetting", b =>
                 {
                     b.Property<int>("Id")

src/NadekoBot/Modules/Administration/GreetBye/GreetService.cs

@@ -606,4 +606,4 @@ public class GreetService : INService, IReadyExecutor
     }
 
     #endregion
-}
+}

src/NadekoBot/Modules/Administration/PermOverrides/DiscordPermOverrideCommands.cs

@@ -14,7 +14,7 @@ public partial class Administration
         [Cmd]
         [RequireContext(ContextType.Guild)]
         [UserPerm(GuildPerm.Administrator)]
-        public async partial Task DiscordPermOverride(CommandOrCrInfo cmd, params GuildPerm[] perms)
+        public async partial Task DiscordPermOverride(CommandOrExprInfo cmd, params GuildPerm[] perms)
         {
             if (perms is null || perms.Length == 0)
             {

src/NadekoBot/Modules/Administration/Self/SelfCommands.cs

@@ -500,14 +500,6 @@ public partial class Administration
             await ReplyConfirmLocalizedAsync(strs.message_sent);
         }
 
-        [Cmd]
-        [OwnerOnly]
-        public async partial Task ImagesReload()
-        {
-            await _service.ReloadImagesAsync();
-            await ReplyConfirmLocalizedAsync(strs.images_loading);
-        }
-
         [Cmd]
         [OwnerOnly]
         public async partial Task StringsReload()

src/NadekoBot/Modules/Administration/Self/SelfService.cs

@@ -20,7 +20,6 @@ public sealed class SelfService : IExecNoCommand, IReadyExecutor, INService
 
     private ConcurrentDictionary<ulong?, ConcurrentDictionary<int, Timer>> autoCommands = new();
 
-    private readonly IImageCache _imgs;
     private readonly IHttpClientFactory _httpFactory;
     private readonly BotConfigService _bss;
     private readonly IPubSub _pubSub;
@@ -28,7 +27,6 @@ public sealed class SelfService : IExecNoCommand, IReadyExecutor, INService
 
     //keys
     private readonly TypedKey<ActivityPubData> _activitySetKey;
-    private readonly TypedKey<bool> _imagesReloadKey;
     private readonly TypedKey<string> _guildLeaveKey;
 
     public SelfService(
@@ -37,7 +35,6 @@ public sealed class SelfService : IExecNoCommand, IReadyExecutor, INService
         DbService db,
         IBotStrings strings,
         IBotCredentials creds,
-        IDataCache cache,
         IHttpClientFactory factory,
         BotConfigService bss,
         IPubSub pubSub,
@@ -48,20 +45,15 @@ public sealed class SelfService : IExecNoCommand, IReadyExecutor, INService
         _strings = strings;
         _client = client;
         _creds = creds;
-        _imgs = cache.LocalImages;
         _httpFactory = factory;
         _bss = bss;
         _pubSub = pubSub;
         _eb = eb;
         _activitySetKey = new("activity.set");
-        _imagesReloadKey = new("images.reload");
         _guildLeaveKey = new("guild.leave");
 
         HandleStatusChanges();
 
-        if (_client.ShardId == 0)
-            _pubSub.Sub(_imagesReloadKey, async _ => await _imgs.Reload());
-
         _pubSub.Sub(_guildLeaveKey,
             async input =>
             {
@@ -325,9 +317,6 @@ public sealed class SelfService : IExecNoCommand, IReadyExecutor, INService
         uow.SaveChanges();
     }
 
-    public Task ReloadImagesAsync()
-        => _pubSub.Pub(_imagesReloadKey, true);
-
     public bool ForwardMessages()
     {
         var isForwarding = false;

src/NadekoBot/Modules/Administration/VcRole/VcRoleService.cs

@@ -8,7 +8,7 @@ namespace NadekoBot.Modules.Administration.Services;
 public class VcRoleService : INService
 {
     public ConcurrentDictionary<ulong, ConcurrentDictionary<ulong, IRole>> VcRoles { get; }
-    public ConcurrentDictionary<ulong, ConcurrentQueue<(bool, IGuildUser, IRole)>> ToAssign { get; }
+    public ConcurrentDictionary<ulong, System.Collections.Concurrent.ConcurrentQueue<(bool, IGuildUser, IRole)>> ToAssign { get; }
     private readonly DbService _db;
     private readonly DiscordSocketClient _client;
 
@@ -37,7 +37,7 @@ public class VcRoleService : INService
         {
             while (true)
             {
-                Task Selector(ConcurrentQueue<(bool, IGuildUser, IRole)> queue)
+                Task Selector(System.Collections.Concurrent.ConcurrentQueue<(bool, IGuildUser, IRole)> queue)
                 {
                     return Task.Run(async () =>
                     {
@@ -203,7 +203,7 @@ public class VcRoleService : INService
 
     private void Assign(bool v, SocketGuildUser gusr, IRole role)
     {
-        var queue = ToAssign.GetOrAdd(gusr.Guild.Id, new ConcurrentQueue<(bool, IGuildUser, IRole)>());
+        var queue = ToAssign.GetOrAdd(gusr.Guild.Id, new System.Collections.Concurrent.ConcurrentQueue<(bool, IGuildUser, IRole)>());
         queue.Enqueue((v, gusr, role));
     }
 }

src/NadekoBot/Modules/CustomReactions/NadekoExpressionsService.cs

@@ -189,7 +189,7 @@ public sealed class NadekoExpressionsService : IExecOnMessage, IReadyExecutor
                     continue;
                 }
 
-                // if CA is disabled, and CR has AllowTarget, then the
+                // if CA is disabled, and expr has AllowTarget, then the
                 // content has to start with the trigger followed by a space
                 if (expr.AllowTarget
                     && content.StartsWith(trigger, StringComparison.OrdinalIgnoreCase)

src/NadekoBot/Modules/Gambling/Bank/BankService.cs

@@ -75,19 +75,4 @@ public sealed class BankService : IBankService, INService
                ?.Balance
                ?? 0;
     }
-
-    public async Task<long> BurnAllAsync(ulong userId)
-    {
-        await using var ctx = _db.GetDbContext();
-        var output = await ctx.GetTable<BankUser>()
-                              .Where(x => x.UserId == userId)
-                              .UpdateWithOutputAsync(old => new()
-                              {
-                                  Balance = 0
-                              });
-        if (output.Length == 0)
-            return 0;
-        
-        return output[0].Deleted.Balance;
-    }
 }

src/NadekoBot/Modules/Gambling/Bank/IBankService.cs

@@ -5,5 +5,4 @@ public interface IBankService
     Task<bool> DepositAsync(ulong userId, long amount);
     Task<bool> WithdrawAsync(ulong userId, long amount);
     Task<long> GetBalanceAsync(ulong userId);
-    Task<long> BurnAllAsync(ulong userId);
 }

src/NadekoBot/Modules/Gambling/DiceRoll/DiceRollCommands.cs

@@ -19,8 +19,8 @@ public partial class Gambling
         private static readonly char[] _fateRolls = { '-', ' ', '+' };
         private readonly IImageCache _images;
 
-        public DiceRollCommands(IDataCache data)
-            => _images = data.LocalImages;
+        public DiceRollCommands(ImageCache images)
+            => _images = images;
 
         [Cmd]
         public async partial Task Roll()
@@ -31,10 +31,10 @@ public partial class Gambling
             var num1 = gen / 10;
             var num2 = gen % 10;
 
-            using var img1 = GetDice(num1);
-            using var img2 = GetDice(num2);
+            using var img1 = await GetDiceAsync(num1);
+            using var img2 = await GetDiceAsync(num2);
             using var img = new[] { img1, img2 }.Merge(out var format);
-            await using var ms = img.ToStream(format);
+            await using var ms = await img.ToStreamAsync(format);
             await ctx.Channel.SendFileAsync(ms,
                 $"dice.{format.FileExtensions.First()}",
                 Format.Bold(ctx.User.ToString()) + " " + GetText(strs.dice_rolled(Format.Code(gen.ToString()))));
@@ -96,7 +96,7 @@ public partial class Gambling
                 else
                     toInsert = dice.Count;
 
-                dice.Insert(toInsert, GetDice(randomNumber));
+                dice.Insert(toInsert, await GetDiceAsync(randomNumber));
                 values.Insert(toInsert, randomNumber);
             }
 
@@ -195,20 +195,19 @@ public partial class Gambling
             await ReplyConfirmLocalizedAsync(strs.dice_rolled(Format.Bold(rolled.ToString())));
         }
 
-        private Image<Rgba32> GetDice(int num)
+        private async Task<Image<Rgba32>> GetDiceAsync(int num)
         {
             if (num is < 0 or > 10)
                 throw new ArgumentOutOfRangeException(nameof(num));
 
             if (num == 10)
             {
-                var images = _images.Dice;
-                using var imgOne = Image.Load(images[1]);
-                using var imgZero = Image.Load(images[0]);
+                using var imgOne = Image.Load(await _images.GetDiceAsync(1));
+                using var imgZero = Image.Load(await _images.GetDiceAsync(0));
                 return new[] { imgOne, imgZero }.Merge();
             }
 
-            return Image.Load(_images.Dice[num]);
+            return Image.Load(await _images.GetDiceAsync(num));
         }
     }
 }

src/NadekoBot/Modules/Gambling/Draw/DrawCommands.cs

@@ -14,8 +14,8 @@ public partial class Gambling
         private static readonly ConcurrentDictionary<IGuild, Deck> _allDecks = new();
         private readonly IImageCache _images;
 
-        public DrawCommands(IDataCache data)
-            => _images = data.LocalImages;
+        public DrawCommands(IImageCache images)
+            => _images = images;
 
         private async Task<(Stream ImageStream, string ToSend)> InternalDraw(int num, ulong? guildId = null)
         {
@@ -43,7 +43,8 @@ public partial class Gambling
 
                 var currentCard = cards.Draw();
                 cardObjects.Add(currentCard);
-                images.Add(Image.Load(_images.GetCard(currentCard.ToString().ToLowerInvariant().Replace(' ', '_'))));
+                var cardName = currentCard.ToString().ToLowerInvariant().Replace(' ', '_');
+                images.Add(Image.Load(await File.ReadAllBytesAsync($"data/images/cards/{cardName}.png")));
             }
 
             using var img = images.Merge();

src/NadekoBot/Modules/Gambling/Events/GameStatusEvent.cs

@@ -1,5 +1,6 @@
 #nullable disable
 using NadekoBot.Services.Database.Models;
+using System.Collections.Concurrent;
 
 namespace NadekoBot.Modules.Gambling.Common.Events;
 

src/NadekoBot/Modules/Gambling/Events/ReactionEvent.cs

@@ -20,7 +20,7 @@ public class ReactionEvent : ICurrencyEvent
     private readonly bool _isPotLimited;
     private readonly ITextChannel _channel;
     private readonly ConcurrentHashSet<ulong> _awardedUsers = new();
-    private readonly ConcurrentQueue<ulong> _toAward = new();
+    private readonly System.Collections.Concurrent.ConcurrentQueue<ulong> _toAward = new();
     private readonly Timer _t;
     private readonly Timer _timeout;
     private readonly bool _noRecentlyJoinedServer;

src/NadekoBot/Modules/Gambling/FlipCoin/FlipCoinCommands.cs

@@ -25,11 +25,17 @@ public partial class Gambling
         private static readonly NadekoRandom _rng = new();
         private readonly IImageCache _images;
         private readonly ICurrencyService _cs;
+        private readonly ImagesConfig _ic;
 
-        public FlipCoinCommands(IDataCache data, ICurrencyService cs, GamblingConfigService gss)
+        public FlipCoinCommands(
+            IImageCache images,
+            ImagesConfig ic,
+            ICurrencyService cs,
+            GamblingConfigService gss)
             : base(gss)
         {
-            _images = data.LocalImages;
+            _ic = ic;
+            _images = images;
             _cs = cs;
         }
 
@@ -47,8 +53,8 @@ public partial class Gambling
             var imgs = new Image<Rgba32>[count];
             for (var i = 0; i < count; i++)
             {
-                var headsArr = _images.Heads[_rng.Next(0, _images.Heads.Count)];
-                var tailsArr = _images.Tails[_rng.Next(0, _images.Tails.Count)];
+                var headsArr = await _images.GetHeadsImageAsync();
+                var tailsArr = await _images.GetTailsImageAsync();
                 if (_rng.Next(0, 10) < 5)
                 {
                     imgs[i] = Image.Load(headsArr);
@@ -94,7 +100,7 @@ public partial class Gambling
 
             BetFlipGuess result;
             Uri imageToSend;
-            var coins = _images.ImageUrls.Coins;
+            var coins = _ic.Data.Coins;
             if (_rng.Next(0, 1000) <= 499)
             {
                 imageToSend = coins.Heads[_rng.Next(0, coins.Heads.Length)];

src/NadekoBot/Modules/Gambling/Gambling.cs

@@ -38,7 +38,6 @@ public partial class Gambling : GamblingModule<GamblingService>
 
     private readonly DbService _db;
     private readonly ICurrencyService _cs;
-    private readonly IDataCache _cache;
     private readonly DiscordSocketClient _client;
     private readonly NumberFormatInfo _enUsCulture;
     private readonly DownloadTracker _tracker;
@@ -51,7 +50,6 @@ public partial class Gambling : GamblingModule<GamblingService>
     public Gambling(
         DbService db,
         ICurrencyService currency,
-        IDataCache cache,
         DiscordSocketClient client,
         DownloadTracker tracker,
         GamblingConfigService configService,
@@ -61,7 +59,6 @@ public partial class Gambling : GamblingModule<GamblingService>
     {
         _db = db;
         _cs = currency;
-        _cache = cache;
         _client = client;
         _bank = bank;
         _ps = ps;
@@ -124,15 +121,17 @@ public partial class Gambling : GamblingModule<GamblingService>
             return;
         }
 
-        if (_cache.AddTimelyClaim(ctx.User.Id, period) is { } rem)
+        if (await _service.ClaimTimelyAsync(ctx.User.Id, period) is { } rem)
         {
-            await ReplyErrorLocalizedAsync(strs.timely_already_claimed(rem.ToString(@"dd\d\ hh\h\ mm\m\ ss\s")));
+            var now = DateTime.UtcNow;
+            var relativeTag = TimestampTag.FromDateTime(now.Add(rem), TimestampTagStyles.Relative);
+            await ReplyErrorLocalizedAsync(strs.timely_already_claimed(relativeTag));
             return;
         }
 
         var result = await _ps.TryGetFeatureLimitAsync(_timelyKey, ctx.User.Id, 0);
 
-        val = (int)(val * (1 + (result.Quota * 0.01f)));
+        val = (int)(val * (1 + (result.Quota! * 0.01f)));
 
         await _cs.AddAsync(ctx.User.Id, val, new("timely", "claim"));
 
@@ -143,7 +142,7 @@ public partial class Gambling : GamblingModule<GamblingService>
     [OwnerOnly]
     public async partial Task TimelyReset()
     {
-        _cache.RemoveAllTimelyClaims();
+        await _service.RemoveAllTimelyClaimsAsync();
         await ReplyConfirmLocalizedAsync(strs.timely_reset);
     }
 

src/NadekoBot/Modules/Gambling/GamblingService.cs

@@ -1,16 +1,13 @@
 #nullable disable
 using LinqToDB;
 using LinqToDB.EntityFrameworkCore;
-using Microsoft.EntityFrameworkCore;
 using NadekoBot.Common.ModuleBehaviors;
 using NadekoBot.Db;
 using NadekoBot.Db.Models;
-using NadekoBot.Migrations;
 using NadekoBot.Modules.Gambling.Common;
 using NadekoBot.Modules.Gambling.Common.Connect4;
 using NadekoBot.Modules.Gambling.Common.Slot;
 using NadekoBot.Modules.Gambling.Common.WheelOfFortune;
-using Newtonsoft.Json;
 
 namespace NadekoBot.Modules.Gambling.Services;
 
@@ -22,7 +19,7 @@ public class GamblingService : INService, IReadyExecutor
     private readonly ICurrencyService _cs;
     private readonly Bot _bot;
     private readonly DiscordSocketClient _client;
-    private readonly IDataCache _cache;
+    private readonly IBotCache _cache;
     private readonly GamblingConfigService _gss;
 
     public GamblingService(
@@ -30,7 +27,7 @@ public class GamblingService : INService, IReadyExecutor
         Bot bot,
         ICurrencyService cs,
         DiscordSocketClient client,
-        IDataCache cache,
+        IBotCache cache,
         GamblingConfigService gss)
     {
         _db = db;
@@ -73,6 +70,7 @@ public class GamblingService : INService, IReadyExecutor
         }
     }
 
+    private static readonly TypedKey<long> _curDecayKey = new("currency:last_decay");
     private async Task CurrencyDecayLoopAsync()
     {
         if (_bot.Client.ShardId != 0)
@@ -88,11 +86,16 @@ public class GamblingService : INService, IReadyExecutor
                 if (config.Decay.Percent is <= 0 or > 1 || maxDecay < 0)
                     continue;
 
+                var now = DateTime.UtcNow;
+                
                 await using var uow = _db.GetDbContext();
-                var lastCurrencyDecay = _cache.GetLastCurrencyDecay();
+                var result = await _cache.GetAsync(_curDecayKey);
 
-                if (DateTime.UtcNow - lastCurrencyDecay < TimeSpan.FromHours(config.Decay.HourInterval))
+                if (result.TryPickT0(out var bin, out _)
+                    && (now - DateTime.FromBinary(bin) < TimeSpan.FromHours(config.Decay.HourInterval)))
+                {
                     continue;
+                }
 
                 Log.Information(@"Decaying users' currency - decay: {ConfigDecayPercent}% 
                                     | max: {MaxDecay} 
@@ -115,8 +118,9 @@ public class GamblingService : INService, IReadyExecutor
                                      : old.CurrencyAmount - maxDecay 
                          });
 
-                _cache.SetLastCurrencyDecay();
                 await uow.SaveChangesAsync();
+
+                await _cache.AddAsync(_curDecayKey, now.ToBinary());
             }
             catch (Exception ex)
             {
@@ -161,60 +165,100 @@ public class GamblingService : INService, IReadyExecutor
         return toReturn;
     }
 
+    private static readonly TypedKey<EconomyResult> _ecoKey = new("nadeko:economy");
+
     public async Task<EconomyResult> GetEconomyAsync()
     {
-        if (_cache.TryGetEconomy(out var data))
-        {
-            try
+        var data = await _cache.GetOrAddAsync(_ecoKey,
+            async () =>
             {
-                return JsonConvert.DeserializeObject<EconomyResult>(data);
-            }
-            catch { }
-        }
+                await using var uow = _db.GetDbContext();
+                var cash = uow.DiscordUser.GetTotalCurrency();
+                var onePercent = uow.DiscordUser.GetTopOnePercentCurrency(_client.CurrentUser.Id);
+                decimal planted = uow.PlantedCurrency.AsQueryable().Sum(x => x.Amount);
+                var waifus = uow.WaifuInfo.GetTotalValue();
+                var bot = uow.DiscordUser.GetUserCurrency(_client.CurrentUser.Id);
+                decimal bank = await uow.GetTable<BankUser>()
+                                        .SumAsyncLinqToDB(x => x.Balance);
 
-        decimal cash;
-        decimal onePercent;
-        decimal planted;
-        decimal waifus;
-        decimal bank;
-        long bot;
+                var result = new EconomyResult
+                {
+                    Cash = cash,
+                    Planted = planted,
+                    Bot = bot,
+                    Waifus = waifus,
+                    OnePercent = onePercent,
+                    Bank = bank
+                };
 
-        using (var uow = _db.GetDbContext())
-        {
-            cash = uow.DiscordUser.GetTotalCurrency();
-            onePercent = uow.DiscordUser.GetTopOnePercentCurrency(_client.CurrentUser.Id);
-            planted = uow.PlantedCurrency.AsQueryable().Sum(x => x.Amount);
-            waifus = uow.WaifuInfo.GetTotalValue();
-            bot = uow.DiscordUser.GetUserCurrency(_client.CurrentUser.Id);
-            bank = await uow.GetTable<BankUser>()
-                      .SumAsyncLinqToDB(x => x.Balance);
-        }
+                return result;
+            },
+            TimeSpan.FromMinutes(3));
 
-        var result = new EconomyResult
-        {
-            Cash = cash,
-            Planted = planted,
-            Bot = bot,
-            Waifus = waifus,
-            OnePercent = onePercent,
-            Bank = bank
-        };
-
-        _cache.SetEconomy(JsonConvert.SerializeObject(result));
-        return result;
+        return data;
     }
 
     public Task<WheelOfFortuneGame.Result> WheelOfFortuneSpinAsync(ulong userId, long bet)
         => new WheelOfFortuneGame(userId, bet, _gss.Data, _cs).SpinAsync();
 
 
-    public struct EconomyResult
+    private static readonly SemaphoreSlim _timelyLock = new (1, 1);
+
+    private static TypedKey<Dictionary<ulong, long>> _timelyKey
+        = new("timely:claims");
+    public async Task<TimeSpan?> ClaimTimelyAsync(ulong userId, int period)
     {
-        public decimal Cash { get; set; }
-        public decimal Planted { get; set; }
-        public decimal Waifus { get; set; }
-        public decimal OnePercent { get; set; }
-        public decimal Bank { get; set; }
-        public long Bot { get; set; }
+        if (period == 0)
+            return null;
+
+        await _timelyLock.WaitAsync();
+        try
+        {
+            // get the dictionary from the cache or get a new one
+            var dict = (await _cache.GetOrAddAsync(_timelyKey,
+                () => Task.FromResult(new Dictionary<ulong, long>())))!;
+
+            var now = DateTime.UtcNow;
+            var nowB = now.ToBinary();
+
+            // try to get users last claim
+            if (!dict.TryGetValue(userId, out var lastB))
+                lastB = dict[userId] = now.ToBinary();
+
+            var diff = now - DateTime.FromBinary(lastB);
+
+            // if its now, or too long ago => success
+            if (lastB == nowB || diff > period.Hours())
+            {
+                // update the cache
+                dict[userId] = nowB;
+                await _cache.AddAsync(_timelyKey, dict);
+                
+                return null;
+            }
+            else
+            {
+                // otherwise return the remaining time
+                return period.Hours() - diff;
+            }
+        }
+        finally
+        {
+            await _timelyLock.WaitAsync();
+        }
+    }
+
+    public async Task RemoveAllTimelyClaimsAsync()
+        => await _cache.RemoveAsync(_timelyKey);
+
+
+    public readonly struct EconomyResult
+    {
+        public decimal Cash { get; init; }
+        public decimal Planted { get; init; }
+        public decimal Waifus { get; init; }
+        public decimal OnePercent { get; init; }
+        public decimal Bank { get; init; }
+        public long Bot { get; init; }
     }
 }

src/NadekoBot/Modules/Gambling/PlantPick/PlantPickService.cs

@@ -34,7 +34,7 @@ public class PlantPickService : INService, IExecNoCommand
         DbService db,
         CommandHandler cmd,
         IBotStrings strings,
-        IDataCache cache,
+        IImageCache images,
         FontProvider fonts,
         ICurrencyService cs,
         CommandHandler cmdHandler,
@@ -43,7 +43,7 @@ public class PlantPickService : INService, IExecNoCommand
     {
         _db = db;
         _strings = strings;
-        _images = cache.LocalImages;
+        _images = images;
         _fonts = fonts;
         _cs = cs;
         _cmdHandler = cmdHandler;
@@ -110,30 +110,21 @@ public class PlantPickService : INService, IExecNoCommand
     /// <param name="pass">Optional password to add to top left corner.</param>
     /// <param name="extension">Extension of the file, defaults to png</param>
     /// <returns>Stream of the currency image</returns>
-    public Stream GetRandomCurrencyImage(string pass, out string extension)
+    public async Task<(Stream, string)> GetRandomCurrencyImageAsync(string pass)
     {
-        // get a random currency image bytes
-        var rng = new NadekoRandom();
-        var curImg = _images.Currency[rng.Next(0, _images.Currency.Count)];
+        var curImg = await _images.GetCurrencyImageAsync();
 
         if (string.IsNullOrWhiteSpace(pass))
         {
             // determine the extension
-            using (_ = Image.Load(curImg, out var format))
-            {
-                extension = format.FileExtensions.FirstOrDefault() ?? "png";
-            }
+            using var load = _ = Image.Load(curImg, out var format);
 
             // return the image
-            return curImg.ToStream();
+            return (curImg.ToStream(), format.FileExtensions.FirstOrDefault() ?? "png");
         }
 
         // get the image stream and extension
-        var (s, ext) = AddPassword(curImg, pass);
-        // set the out extension parameter to the extension we've got
-        extension = ext;
-        // return the image
-        return s;
+        return AddPassword(curImg, pass);
     }
 
     /// <summary>
@@ -214,10 +205,10 @@ public class PlantPickService : INService, IExecNoCommand
                         var pw = config.Generation.HasPassword ? GenerateCurrencyPassword().ToUpperInvariant() : null;
 
                         IUserMessage sent;
-                        await using (var stream = GetRandomCurrencyImage(pw, out var ext))
-                        {
+                        var (stream, ext) = await GetRandomCurrencyImageAsync(pw);
+
+                        await using (stream)
                             sent = await channel.SendFileAsync(stream, $"currency_image.{ext}", toSend);
-                        }
 
                         await AddPlantToDatabase(channel.GuildId,
                             channel.Id,
@@ -278,7 +269,7 @@ public class PlantPickService : INService, IExecNoCommand
                 if (amount > 0)
                     // give the picked currency to the user
                     await _cs.AddAsync(uid, amount, new("currency", "collect"));
-                uow.SaveChanges();
+                await uow.SaveChangesAsync();
             }
 
             try
@@ -316,11 +307,14 @@ public class PlantPickService : INService, IExecNoCommand
                 msgToSend += " " + GetText(gid, strs.pick_sn(prefix));
 
             //get the image
-            await using var stream = GetRandomCurrencyImage(pass, out var ext);
+            var (stream, ext) = await GetRandomCurrencyImageAsync(pass);
             // send it
-            var msg = await ch.SendFileAsync(stream, $"img.{ext}", msgToSend);
-            // return sent message's id (in order to be able to delete it when it's picked)
-            return msg.Id;
+            await using (stream)
+            {
+                var msg = await ch.SendFileAsync(stream, $"img.{ext}", msgToSend);
+                // return sent message's id (in order to be able to delete it when it's picked)
+                return msg.Id;
+            }
         }
         catch
         {

src/NadekoBot/Modules/Gambling/Slot/SlotCommands.cs

@@ -32,13 +32,13 @@ public partial class Gambling
         private readonly DbService _db;
 
         public SlotCommands(
-            IDataCache data,
+            ImageCache images,
             FontProvider fonts,
             DbService db,
             GamblingConfigService gamb)
             : base(gamb)
         {
-            _images = data.LocalImages;
+            _images = images;
             _fonts = fonts;
             _db = db;
         }
@@ -130,7 +130,8 @@ public partial class Gambling
                                   ?? 0;
                 }
 
-                using (var bgImage = Image.Load<Rgba32>(_images.SlotBackground, out _))
+                var slotBg = await _images.GetSlotBgAsync();
+                using (var bgImage = Image.Load<Rgba32>(slotBg, out _))
                 {
                     var numbers = new int[3];
                     result.Rolls.CopyTo(numbers, 0);
@@ -184,7 +185,7 @@ public partial class Gambling
 
                     for (var i = 0; i < 3; i++)
                     {
-                        using var img = Image.Load(_images.SlotEmojis[numbers[i]]);
+                        using var img = Image.Load(await _images.GetSlotEmojiAsync(numbers[i]));
                         bgImage.Mutate(x => x.DrawImage(img, new Point(148 + (105 * i), 217), 1f));
                     }
 
@@ -201,7 +202,7 @@ public partial class Gambling
                             msg = GetText(strs.slot_jackpot(30));
                     }
 
-                    await using (var imgStream = bgImage.ToStream())
+                    await using (var imgStream = await bgImage.ToStreamAsync())
                     {
                         await ctx.Channel.SendFileAsync(imgStream,
                             "result.png",

src/NadekoBot/Modules/Gambling/Waifus/WaifuService.cs

@@ -1,5 +1,6 @@
 #nullable disable
 using LinqToDB;
+using LinqToDB.EntityFrameworkCore;
 using Microsoft.EntityFrameworkCore;
 using NadekoBot.Common.ModuleBehaviors;
 using NadekoBot.Db;
@@ -14,7 +15,7 @@ public class WaifuService : INService, IReadyExecutor
 {
     private readonly DbService _db;
     private readonly ICurrencyService _cs;
-    private readonly IDataCache _cache;
+    private readonly IBotCache _cache;
     private readonly GamblingConfigService _gss;
     private readonly IBotCredentials _creds;
     private readonly DiscordSocketClient _client;
@@ -22,7 +23,7 @@ public class WaifuService : INService, IReadyExecutor
     public WaifuService(
         DbService db,
         ICurrencyService cs,
-        IDataCache cache,
+        IBotCache cache,
         GamblingConfigService gss,
         IBotCredentials creds,
         DiscordSocketClient client)
@@ -236,8 +237,13 @@ public class WaifuService : INService, IReadyExecutor
             var newAff = target is null ? null : uow.GetOrCreateUser(target);
             if (w?.Affinity?.UserId == target?.Id)
             {
+                return (null, false, null);
             }
-            else if (!_cache.TryAddAffinityCooldown(user.Id, out remaining))
+
+            remaining = await _cache.GetRatelimitAsync(GetAffinityKey(user.Id),
+                30.Minutes());
+            
+            if (remaining is not null)
             {
             }
             else if (w is null)
@@ -294,6 +300,12 @@ public class WaifuService : INService, IReadyExecutor
         return uow.WaifuInfo.GetWaifuUserId(ownerId, name);
     }
 
+    private static TypedKey<long> GetDivorceKey(ulong userId)
+        => new($"waifu:divorce_cd:{userId}");
+    
+    private static TypedKey<long> GetAffinityKey(ulong userId)
+        => new($"waifu:affinity:{userId}");
+    
     public async Task<(WaifuInfo, DivorceResult, long, TimeSpan?)> DivorceWaifuAsync(IUser user, ulong targetId)
     {
         DivorceResult result;
@@ -305,10 +317,15 @@ public class WaifuService : INService, IReadyExecutor
             w = uow.WaifuInfo.ByWaifuUserId(targetId);
             if (w?.Claimer is null || w.Claimer.UserId != user.Id)
                 result = DivorceResult.NotYourWife;
-            else if (!_cache.TryAddDivorceCooldown(user.Id, out remaining))
-                result = DivorceResult.Cooldown;
             else
             {
+                remaining = await _cache.GetRatelimitAsync(GetDivorceKey(user.Id), 6.Hours());
+                if (remaining is TimeSpan rem)
+                {
+                    result = DivorceResult.Cooldown;
+                    return (w, result, amount, rem);
+                }
+
                 amount = w.Price / 2;
 
                 if (w.Affinity?.UserId == user.Id)
@@ -486,13 +503,13 @@ public class WaifuService : INService, IReadyExecutor
                    .ToList();
     }
 
+    private static readonly TypedKey<long> _waifuDecayKey = $"waifu:last_decay";
     public async Task OnReadyAsync()
     {
         // only decay waifu values from shard 0
         if (_client.ShardId != 0)
             return;
 
-        var redisKey = $"{_creds.RedisKey()}_last_waifu_decay";
         while (true)
         {
             try
@@ -504,28 +521,31 @@ public class WaifuService : INService, IReadyExecutor
                 if (multi is < 0f or > 1f || decayInterval < 0)
                     continue;
 
-                var val = await _cache.Redis.GetDatabase().StringGetAsync(redisKey);
-                if (val != default)
+                var now = DateTime.UtcNow;
+                var nowB = now.ToBinary();
+
+                var result = await _cache.GetAsync(_waifuDecayKey);
+                
+                if (result.TryGetValue(out var val))
                 {
-                    var lastDecay = DateTime.FromBinary((long)val);
+                    var lastDecay = DateTime.FromBinary(val);
                     var toWait = decayInterval.Hours() - (DateTime.UtcNow - lastDecay);
 
                     if (toWait > 0.Hours())
                         continue;
                 }
 
-                await _cache.Redis.GetDatabase().StringSetAsync(redisKey, DateTime.UtcNow.ToBinary());
+                await _cache.AddAsync(_waifuDecayKey, nowB);
 
                 await using var uow = _db.GetDbContext();
 
-                await uow.WaifuInfo
+                await uow.GetTable<WaifuInfo>()
                          .Where(x => x.Price > minPrice && x.ClaimerId == null)
                          .UpdateAsync(old => new()
                          {
                              Price = (long)(old.Price * multi)
                          });
 
-                await uow.SaveChangesAsync();
             }
             catch (Exception ex)
             {

src/NadekoBot/Modules/Games/ChatterBot/ChatterbotService.cs

@@ -136,7 +136,7 @@ public class ChatterBotService : IExecOnMessage
 
             var channel = (ITextChannel)usrMsg.Channel;
             var conf = _ps.GetConfig();
-            if (conf.IsEnabled)
+            if (!_creds.IsOwner(sg.OwnerId) && conf.IsEnabled)
             {
                 var quota = await _ps.TryGetFeatureLimitAsync(_flKey, sg.OwnerId, 0);
 

src/NadekoBot/Modules/Games/Games.cs

@@ -14,9 +14,9 @@ public partial class Games : NadekoModule<GamesService>
     private readonly IHttpClientFactory _httpFactory;
     private readonly Random _rng = new();
 
-    public Games(IDataCache data, IHttpClientFactory factory)
+    public Games(IImageCache images, IHttpClientFactory factory)
     {
-        _images = data.LocalImages;
+        _images = images;
         _httpFactory = factory;
     }
 

src/NadekoBot/Modules/Games/GirlRating.cs

@@ -28,11 +28,12 @@ public class GirlRating
         Roll = roll;
         Advice = advice; // convenient to have it here, even though atm there are only few different ones.
 
-        Stream = new(() =>
+        Stream = new(async () =>
         {
             try
             {
-                using var img = Image.Load(_images.RategirlMatrix);
+                var bgBytes = await _images.GetRategirlBgAsync();
+                using var img = Image.Load(bgBytes);
                 const int minx = 35;
                 const int miny = 385;
                 const int length = 345;
@@ -40,7 +41,8 @@ public class GirlRating
                 var pointx = (int)(minx + (length * (Hot / 10)));
                 var pointy = (int)(miny - (length * ((Crazy - 4) / 6)));
 
-                using (var pointImg = Image.Load(_images.RategirlDot))
+                var dotBytes = await _images.GetRategirlDotAsync(); 
+                using (var pointImg = Image.Load(dotBytes))
                 {
                     img.Mutate(x => x.DrawImage(pointImg, new(pointx - 10, pointy - 10), new GraphicsOptions()));
                 }

src/NadekoBot/Modules/Games/Trivia/TriviaCommands.cs

@@ -9,14 +9,14 @@ public partial class Games
     [Group]
     public partial class TriviaCommands : NadekoModule<GamesService>
     {
-        private readonly IDataCache _cache;
+        private readonly ILocalDataCache _cache;
         private readonly ICurrencyService _cs;
         private readonly GamesConfigService _gamesConfig;
         private readonly DiscordSocketClient _client;
 
         public TriviaCommands(
             DiscordSocketClient client,
-            IDataCache cache,
+            ILocalDataCache cache,
             ICurrencyService cs,
             GamesConfigService gamesConfig)
         {

src/NadekoBot/Modules/Games/Trivia/TriviaGame.cs

@@ -17,7 +17,7 @@ public class TriviaGame
     public bool GameActive { get; private set; }
     public bool ShouldStopGame { get; private set; }
     private readonly SemaphoreSlim _guessLock = new(1, 1);
-    private readonly IDataCache _cache;
+    private readonly ILocalDataCache _cache;
     private readonly IBotStrings _strings;
     private readonly DiscordSocketClient _client;
     private readonly GamesConfig _config;
@@ -35,7 +35,7 @@ public class TriviaGame
         IBotStrings strings,
         DiscordSocketClient client,
         GamesConfig config,
-        IDataCache cache,
+        ILocalDataCache cache,
         ICurrencyService cs,
         IGuild guild,
         ITextChannel channel,
@@ -70,7 +70,7 @@ public class TriviaGame
             showHowToQuit = !showHowToQuit;
 
             // load question
-            CurrentQuestion = _questionPool.GetRandomQuestion(OldQuestions, _options.IsPokemon);
+            CurrentQuestion = await _questionPool.GetRandomQuestionAsync(OldQuestions, _options.IsPokemon);
             if (string.IsNullOrWhiteSpace(CurrentQuestion?.Answer)
                 || string.IsNullOrWhiteSpace(CurrentQuestion.Question))
             {

src/NadekoBot/Modules/Games/Trivia/TriviaQuestion.cs

@@ -4,6 +4,15 @@ using System.Text.RegularExpressions;
 // THANKS @ShoMinamimoto for suggestions and coding help
 namespace NadekoBot.Modules.Games.Common.Trivia;
 
+public sealed class TriviaQuestionModel
+{
+    public string Category { get; init; }
+    public string Question { get; init; }
+    public string ImageUrl { get; init; }
+    public string AnswerImageUrl { get; init; }
+    public string Answer { get; init; }
+}
+
 public class TriviaQuestion
 {
     public const int MAX_STRING_LENGTH = 22;
@@ -17,29 +26,30 @@ public class TriviaQuestion
         new(22, 3)
     };
 
-    public string Category { get; set; }
-    public string Question { get; set; }
-    public string ImageUrl { get; set; }
-    public string AnswerImageUrl { get; set; }
-    public string Answer { get; set; }
+    public string Category
+        => _qModel.Category;
+
+    public string Question
+        => _qModel.Question;
+
+    public string ImageUrl
+        => _qModel.ImageUrl;
+
+    public string AnswerImageUrl
+        => _qModel.AnswerImageUrl ?? ImageUrl;
+
+    public string Answer
+        => _qModel.Answer;
 
     public string CleanAnswer
         => cleanAnswer ?? (cleanAnswer = Clean(Answer));
 
     private string cleanAnswer;
+    private readonly TriviaQuestionModel _qModel;
 
-    public TriviaQuestion(
-        string q,
-        string a,
-        string c,
-        string img = null,
-        string answerImage = null)
+    public TriviaQuestion(TriviaQuestionModel qModel)
     {
-        Question = q;
-        Answer = a;
-        Category = c;
-        ImageUrl = img;
-        AnswerImageUrl = answerImage ?? img;
+        _qModel = qModel;
     }
 
     public string GetHint()

src/NadekoBot/Modules/Games/Trivia/TriviaQuestionPool.cs

@@ -1,45 +1,48 @@
-#nullable disable
 namespace NadekoBot.Modules.Games.Common.Trivia;
 
 public class TriviaQuestionPool
 {
-    private TriviaQuestion[] Pool
-        => _cache.LocalData.TriviaQuestions;
-
-    private IReadOnlyDictionary<int, string> Map
-        => _cache.LocalData.PokemonMap;
-
-    private readonly IDataCache _cache;
+    private readonly ILocalDataCache _cache;
     private readonly int _maxPokemonId;
 
     private readonly NadekoRandom _rng = new();
 
-    public TriviaQuestionPool(IDataCache cache)
+    public TriviaQuestionPool(ILocalDataCache cache)
     {
         _cache = cache;
         _maxPokemonId = 721; //xd
     }
 
-    public TriviaQuestion GetRandomQuestion(HashSet<TriviaQuestion> exclude, bool isPokemon)
+    public async Task<TriviaQuestion?> GetRandomQuestionAsync(HashSet<TriviaQuestion> exclude, bool isPokemon)
     {
-        if (Pool.Length == 0)
-            return null;
-
         if (isPokemon)
         {
+            var pokes = await _cache.GetPokemonMapAsync();
+
+            if (pokes is null or { Length: 0 })
+                return default;
+            
             var num = _rng.Next(1, _maxPokemonId + 1);
-            return new("Who's That Pokémon?",
-                Map[num].ToTitleCase(),
-                "Pokemon",
-                $@"https://nadeko.bot/images/pokemon/shadows/{num}.png",
-                $@"https://nadeko.bot/images/pokemon/real/{num}.png");
+            return new(new()
+            {
+                Question = "Who's That Pokémon?",
+                Answer = pokes[num].Name.ToTitleCase(),
+                Category = "Pokemon",
+                ImageUrl = $@"https://nadeko.bot/images/pokemon/shadows/{num}.png",
+                AnswerImageUrl = $@"https://nadeko.bot/images/pokemon/real/{num}.png"
+            });
         }
 
         TriviaQuestion randomQuestion;
-        while (exclude.Contains(randomQuestion = Pool[_rng.Next(0, Pool.Length)]))
+        var pool = await _cache.GetTriviaQuestionsAsync();
+        
+        if(pool is null)
+            return default;
+        
+        while (exclude.Contains(randomQuestion = new(pool[_rng.Next(0, pool.Length)])))
         {
             // if too many questions are excluded, clear the exclusion list and start over
-            if (exclude.Count > Pool.Length / 10 * 9)
+            if (exclude.Count > pool.Length / 10 * 9)
             {
                 exclude.Clear();
                 break;

src/NadekoBot/Modules/Help/Help.cs

@@ -145,7 +145,7 @@ public partial class Help : NadekoModule<HelpService>
                 return "❓";
             case "administration":
                 return "🛠️";
-            case "customreactions":
+            case "expressions":
                 return "🗣️";
             case "searches":
                 return "🔍";
@@ -402,7 +402,7 @@ public partial class Help : NadekoModule<HelpService>
                     ContentType = "application/json",
                     ContentBody = uploadData,
                     // either use a path provided in the argument or the default one for public nadeko, other/cmds.json
-                    Key = $"cmds/v4/{StatsService.BOT_VERSION}.json",
+                    Key = $"cmds/{StatsService.BOT_VERSION}.json",
                     CannedACL = S3CannedACL.PublicRead
                 });
             }
@@ -414,7 +414,7 @@ public partial class Help : NadekoModule<HelpService>
                 using var oldVersionObject = await dlClient.GetObjectAsync(new()
                 {
                     BucketName = "nadeko-pictures",
-                    Key = "cmds/v4/versions.json"
+                    Key = "cmds/versions.json"
                 });
 
                 await using var ms = new MemoryStream();
@@ -445,7 +445,7 @@ public partial class Help : NadekoModule<HelpService>
                     ContentType = "application/json",
                     ContentBody = versionListString,
                     // either use a path provided in the argument or the default one for public nadeko, other/cmds.json
-                    Key = "cmds/v4/versions.json",
+                    Key = "cmds/versions.json",
                     CannedACL = S3CannedACL.PublicRead
                 });
             }

src/NadekoBot/Modules/Music/_Common/Impl/RedisTrackCacher.cs

@@ -1,209 +0,0 @@
-using StackExchange.Redis;
-using System.Runtime.CompilerServices;
-using System.Text.Json;
-
-namespace NadekoBot.Modules.Music;
-
-public sealed class RedisTrackCacher : ITrackCacher
-{
-    private readonly ConnectionMultiplexer _multiplexer;
-
-    public RedisTrackCacher(ConnectionMultiplexer multiplexer)
-        => _multiplexer = multiplexer;
-
-    public async Task<string?> GetOrCreateStreamLink(
-        string id,
-        MusicPlatform platform,
-        Func<Task<(string StreamUrl, TimeSpan Expiry)>> streamUrlFactory)
-    {
-        var trackStreamKey = CreateStreamKey(id, platform);
-
-        var value = await GetStreamFromCacheInternalAsync(trackStreamKey);
-
-        // if there is no cached value
-        if (value == default)
-        {
-            // otherwise retrieve and cache a new value, and run this method again
-            var success = await CreateAndCacheStreamUrlAsync(trackStreamKey, streamUrlFactory);
-            if (!success)
-                return null;
-
-            return await GetOrCreateStreamLink(id, platform, streamUrlFactory);
-        }
-
-        // cache new one for future use
-        _ = Task.Run(() => CreateAndCacheStreamUrlAsync(trackStreamKey, streamUrlFactory));
-
-        return value;
-    }
-
-
-    [MethodImpl(MethodImplOptions.AggressiveInlining)]
-    private static string CreateStreamKey(string id, MusicPlatform platform)
-        => $"track:stream:{platform}:{id}";
-
-    private async Task<bool> CreateAndCacheStreamUrlAsync(
-        string trackStreamKey,
-        Func<Task<(string StreamUrl, TimeSpan Expiry)>> factory)
-    {
-        try
-        {
-            var data = await factory();
-            if (data == default)
-                return false;
-
-            await CacheStreamUrlInternalAsync(trackStreamKey, data.StreamUrl, data.Expiry);
-            return true;
-        }
-        catch (Exception ex)
-        {
-            Log.Error(ex, "Error resolving stream link for {TrackCacheKey}", trackStreamKey);
-            return false;
-        }
-    }
-
-    public Task CacheStreamUrlAsync(
-        string id,
-        MusicPlatform platform,
-        string url,
-        TimeSpan expiry)
-        => CacheStreamUrlInternalAsync(CreateStreamKey(id, platform), url, expiry);
-
-    private async Task CacheStreamUrlInternalAsync(string trackStreamKey, string url, TimeSpan expiry)
-    {
-        // keys need to be expired after an hour
-        // to make sure client doesn't get an expired stream url
-        // to achieve this, track keys will be just pointers to real data
-        // but that data will expire
-
-        var db = _multiplexer.GetDatabase();
-        var dataKey = $"entry:{Guid.NewGuid()}:{trackStreamKey}";
-        await db.StringSetAsync(dataKey, url, expiry);
-        await db.ListRightPushAsync(trackStreamKey, dataKey);
-    }
-
-    private async Task<string?> GetStreamFromCacheInternalAsync(string trackStreamKey)
-    {
-        // Job of the method which retrieves keys is to pop the elements
-        // from the list of cached trackurls until it finds a non-expired key
-
-        var db = _multiplexer.GetDatabase();
-        while (true)
-        {
-            string? dataKey = await db.ListLeftPopAsync(trackStreamKey);
-            if (dataKey == default)
-                return null;
-
-            var streamUrl = await db.StringGetAsync(dataKey);
-            if (streamUrl == default)
-                continue;
-
-            return streamUrl;
-        }
-    }
-
-    [MethodImpl(MethodImplOptions.AggressiveInlining)]
-    private static string CreateCachedDataKey(string id, MusicPlatform platform)
-        => $"track:data:{platform}:{id}";
-
-    public Task CacheTrackDataAsync(ICachableTrackData data)
-    {
-        var db = _multiplexer.GetDatabase();
-
-        var trackDataKey = CreateCachedDataKey(data.Id, data.Platform);
-        var dataString = JsonSerializer.Serialize((object)data);
-        // cache for 1 day
-        return db.StringSetAsync(trackDataKey, dataString, TimeSpan.FromDays(1));
-    }
-
-    public async Task<ICachableTrackData?> GetCachedDataByIdAsync(string id, MusicPlatform platform)
-    {
-        var db = _multiplexer.GetDatabase();
-
-        var trackDataKey = CreateCachedDataKey(id, platform);
-        var data = await db.StringGetAsync(trackDataKey);
-        if (data == default)
-            return null;
-
-        return JsonSerializer.Deserialize<CachableTrackData>(data);
-    }
-
-    [MethodImpl(MethodImplOptions.AggressiveInlining)]
-    private static string CreateCachedQueryDataKey(string query, MusicPlatform platform)
-        => $"track:query_to_id:{platform}:{query}";
-
-    public async Task<ICachableTrackData?> GetCachedDataByQueryAsync(string query, MusicPlatform platform)
-    {
-        query = Uri.EscapeDataString(query.Trim());
-
-        var db = _multiplexer.GetDatabase();
-        var queryDataKey = CreateCachedQueryDataKey(query, platform);
-
-        var trackId = await db.StringGetAsync(queryDataKey);
-        if (trackId == default)
-            return null;
-
-        return await GetCachedDataByIdAsync(trackId, platform);
-    }
-
-    public async Task CacheTrackDataByQueryAsync(string query, ICachableTrackData data)
-    {
-        query = Uri.EscapeDataString(query.Trim());
-
-        // first cache the data
-        await CacheTrackDataAsync(data);
-
-        // then map the query to cached data's id
-        var db = _multiplexer.GetDatabase();
-
-        var queryDataKey = CreateCachedQueryDataKey(query, data.Platform);
-        await db.StringSetAsync(queryDataKey, data.Id, TimeSpan.FromDays(7));
-    }
-
-    [MethodImpl(MethodImplOptions.AggressiveInlining)]
-    private static string CreateCachedPlaylistKey(string playlistId, MusicPlatform platform)
-        => $"playlist:{platform}:{playlistId}";
-
-    public async Task<IReadOnlyCollection<string>> GetPlaylistTrackIdsAsync(string playlistId, MusicPlatform platform)
-    {
-        var db = _multiplexer.GetDatabase();
-        var key = CreateCachedPlaylistKey(playlistId, platform);
-        var vals = await db.ListRangeAsync(key);
-        if (vals == default || vals.Length == 0)
-            return Array.Empty<string>();
-
-        return vals.Select(x => x.ToString()).ToList();
-    }
-
-    public async Task CachePlaylistTrackIdsAsync(string playlistId, MusicPlatform platform, IEnumerable<string> ids)
-    {
-        var db = _multiplexer.GetDatabase();
-        var key = CreateCachedPlaylistKey(playlistId, platform);
-        await db.ListRightPushAsync(key, ids.Select(x => (RedisValue)x).ToArray());
-        await db.KeyExpireAsync(key, TimeSpan.FromDays(7));
-    }
-
-    [MethodImpl(MethodImplOptions.AggressiveInlining)]
-    private static string CreateCachedPlaylistQueryKey(string query, MusicPlatform platform)
-        => $"playlist:query:{platform}:{query}";
-
-    public Task CachePlaylistIdByQueryAsync(string query, MusicPlatform platform, string playlistId)
-    {
-        query = Uri.EscapeDataString(query.Trim());
-        var key = CreateCachedPlaylistQueryKey(query, platform);
-        var db = _multiplexer.GetDatabase();
-        return db.StringSetAsync(key, playlistId, TimeSpan.FromDays(7));
-    }
-
-    public async Task<string?> GetPlaylistIdByQueryAsync(string query, MusicPlatform platform)
-    {
-        query = Uri.EscapeDataString(query.Trim());
-        var key = CreateCachedPlaylistQueryKey(query, platform);
-
-        var val = await _multiplexer.GetDatabase().StringGetAsync(key);
-        if (val == default)
-            return null;
-
-        return val;
-    }
-}

src/NadekoBot/Modules/Music/_Common/Impl/TrackCacher.cs

@@ -0,0 +1,105 @@
+namespace NadekoBot.Modules.Music;
+
+public sealed class TrackCacher : ITrackCacher
+{
+    private readonly IBotCache _cache;
+
+    public TrackCacher(IBotCache cache)
+        => _cache = cache;
+
+
+    private TypedKey<string> GetStreamLinkKey(MusicPlatform platform, string id)
+        => new($"music:stream:{platform}:{id}");
+    
+    public async Task<string?> GetOrCreateStreamLink(
+        string id,
+        MusicPlatform platform,
+        Func<Task<(string StreamUrl, TimeSpan Expiry)>> streamUrlFactory)
+    {
+        var key = GetStreamLinkKey(platform, id);
+
+        var streamUrl = await _cache.GetOrDefaultAsync(key);
+        await _cache.RemoveAsync(key);
+        
+        if (streamUrl == default)
+        {
+            (streamUrl, _) = await streamUrlFactory();
+        }
+
+        // make a new one for later use
+        _ = Task.Run(async () =>
+        {
+            (streamUrl, var expiry) = await streamUrlFactory();
+            await CacheStreamUrlAsync(id, platform, streamUrl, expiry);
+        });
+        
+        return streamUrl;
+    }
+
+    public async Task CacheStreamUrlAsync(
+        string id,
+        MusicPlatform platform,
+        string url,
+        TimeSpan expiry)
+        => await _cache.AddAsync(GetStreamLinkKey(platform, id), url, expiry);
+
+    // track data by id
+    private TypedKey<CachableTrackData> GetTrackDataKey(MusicPlatform platform, string id)
+        => new($"music:track:{platform}:{id}");
+    public async Task CacheTrackDataAsync(ICachableTrackData data)
+        => await _cache.AddAsync(GetTrackDataKey(data.Platform, data.Id), ToCachableTrackData(data));
+
+    private CachableTrackData ToCachableTrackData(ICachableTrackData data)
+        => new CachableTrackData()
+        {
+            Id = data.Id,
+            Platform = data.Platform,
+            Thumbnail = data.Thumbnail,
+            Title = data.Title,
+            Url = data.Url,
+        };
+
+    public async Task<ICachableTrackData?> GetCachedDataByIdAsync(string id, MusicPlatform platform)
+        => await _cache.GetOrDefaultAsync(GetTrackDataKey(platform, id)); 
+    
+    
+    // track data by query
+    private TypedKey<CachableTrackData> GetTrackDataQueryKey(MusicPlatform platform, string query)
+        => new($"music:track:{platform}:q:{query}");
+
+    public async Task CacheTrackDataByQueryAsync(string query, ICachableTrackData data)
+        => await Task.WhenAll(
+            _cache.AddAsync(GetTrackDataQueryKey(data.Platform, query), ToCachableTrackData(data)).AsTask(),
+            _cache.AddAsync(GetTrackDataKey(data.Platform, data.Id), ToCachableTrackData(data)).AsTask());
+    
+    public async Task<ICachableTrackData?> GetCachedDataByQueryAsync(string query, MusicPlatform platform)
+        => await _cache.GetOrDefaultAsync(GetTrackDataQueryKey(platform, query));
+
+
+    // playlist track ids by playlist id
+    private TypedKey<List<string>> GetPlaylistTracksCacheKey(string playlist, MusicPlatform platform)
+        => new($"music:playlist_tracks:{platform}:{playlist}");
+
+    public async Task CachePlaylistTrackIdsAsync(string playlistId, MusicPlatform platform, IEnumerable<string> ids)
+        => await _cache.AddAsync(GetPlaylistTracksCacheKey(playlistId, platform), ids.ToList());
+
+    public async Task<IReadOnlyCollection<string>> GetPlaylistTrackIdsAsync(string playlistId, MusicPlatform platform)
+    {
+        var result = await _cache.GetAsync(GetPlaylistTracksCacheKey(playlistId, platform));
+        if (result.TryGetValue(out var val))
+            return val;
+
+        return Array.Empty<string>();
+    }
+
+
+    // playlist id by query
+    private TypedKey<string> GetPlaylistCacheKey(string query, MusicPlatform platform)
+        => new($"music:playlist_id:{platform}:{query}");
+    
+    public async Task CachePlaylistIdByQueryAsync(string query, MusicPlatform platform, string playlistId)
+        => await _cache.AddAsync(GetPlaylistCacheKey(query, platform), playlistId);
+
+    public async Task<string?> GetPlaylistIdByQueryAsync(string query, MusicPlatform platform)
+        => await _cache.GetOrDefaultAsync(GetPlaylistCacheKey(query, platform));
+}

src/NadekoBot/Modules/Permissions/CommandCooldown/CmdCdsCommands.cs

@@ -79,7 +79,7 @@ public partial class Permissions
         [Cmd]
         [RequireContext(ContextType.Guild)]
         [Priority(1)]
-        public partial Task CmdCooldown(CommandOrCrInfo command, int secs)
+        public partial Task CmdCooldown(CommandOrExprInfo command, int secs)
             => CmdCooldownInternal(command.Name, secs);
 
         [Cmd]

src/NadekoBot/Modules/Permissions/GlobalPermissions/GlobalPermissionCommands.cs

@@ -60,7 +60,7 @@ public partial class Permissions
 
         [Cmd]
         [OwnerOnly]
-        public async partial Task GlobalCommand(CommandOrCrInfo cmd)
+        public async partial Task GlobalCommand(CommandOrExprInfo cmd)
         {
             var commandName = cmd.Name.ToLowerInvariant();
             var added = _service.ToggleCommand(commandName);

src/NadekoBot/Modules/Permissions/Permissions.cs

@@ -204,7 +204,7 @@ public partial class Permissions : NadekoModule<PermissionService>
 
     [Cmd]
     [RequireContext(ContextType.Guild)]
-    public async partial Task SrvrCmd(CommandOrCrInfo command, PermissionAction action)
+    public async partial Task SrvrCmd(CommandOrExprInfo command, PermissionAction action)
     {
         await _service.AddPermissions(ctx.Guild.Id,
             new Permissionv2
@@ -245,7 +245,7 @@ public partial class Permissions : NadekoModule<PermissionService>
 
     [Cmd]
     [RequireContext(ContextType.Guild)]
-    public async partial Task UsrCmd(CommandOrCrInfo command, PermissionAction action, [Leftover] IGuildUser user)
+    public async partial Task UsrCmd(CommandOrExprInfo command, PermissionAction action, [Leftover] IGuildUser user)
     {
         await _service.AddPermissions(ctx.Guild.Id,
             new Permissionv2
@@ -302,7 +302,7 @@ public partial class Permissions : NadekoModule<PermissionService>
 
     [Cmd]
     [RequireContext(ContextType.Guild)]
-    public async partial Task RoleCmd(CommandOrCrInfo command, PermissionAction action, [Leftover] IRole role)
+    public async partial Task RoleCmd(CommandOrExprInfo command, PermissionAction action, [Leftover] IRole role)
     {
         if (role == role.Guild.EveryoneRole)
             return;
@@ -366,7 +366,7 @@ public partial class Permissions : NadekoModule<PermissionService>
 
     [Cmd]
     [RequireContext(ContextType.Guild)]
-    public async partial Task ChnlCmd(CommandOrCrInfo command, PermissionAction action, [Leftover] ITextChannel chnl)
+    public async partial Task ChnlCmd(CommandOrExprInfo command, PermissionAction action, [Leftover] ITextChannel chnl)
     {
         await _service.AddPermissions(ctx.Guild.Id,
             new Permissionv2

src/NadekoBot/Modules/Searches/Anime/AnimeResult.cs

@@ -1,34 +1,38 @@
 #nullable disable
-using Newtonsoft.Json;
+using System.Text.Json.Serialization;
 
 namespace NadekoBot.Modules.Searches.Common;
 
 public class AnimeResult
 {
+    [JsonPropertyName("id")]
     public int Id { get; set; }
 
-    public string AiringStatus
-        => AiringStatusParsed.ToTitleCase();
-
-    [JsonProperty("airing_status")]
+    [JsonPropertyName("airing_status")]
     public string AiringStatusParsed { get; set; }
 
-    [JsonProperty("title_english")]
+    [JsonPropertyName("title_english")]
     public string TitleEnglish { get; set; }
 
-    [JsonProperty("total_episodes")]
+    [JsonPropertyName("total_episodes")]
     public int TotalEpisodes { get; set; }
-
+    
+    [JsonPropertyName("description")]
     public string Description { get; set; }
 
-    [JsonProperty("image_url_lge")]
+    [JsonPropertyName("image_url_lge")]
     public string ImageUrlLarge { get; set; }
 
+    [JsonPropertyName("genres")]
     public string[] Genres { get; set; }
 
-    [JsonProperty("average_score")]
-    public string AverageScore { get; set; }
+    [JsonPropertyName("average_score")]
+    public float AverageScore { get; set; }
 
+    
+    public string AiringStatus
+        => AiringStatusParsed.ToTitleCase();
+    
     public string Link
         => "http://anilist.co/anime/" + Id;
 

src/NadekoBot/Modules/Searches/Anime/AnimeSearchService.cs

@@ -1,17 +1,15 @@
 #nullable disable
-using AngleSharp;
-using AngleSharp.Html.Dom;
 using NadekoBot.Modules.Searches.Common;
-using Newtonsoft.Json;
+using System.Net.Http.Json;
 
 namespace NadekoBot.Modules.Searches.Services;
 
 public class AnimeSearchService : INService
 {
-    private readonly IDataCache _cache;
+    private readonly IBotCache _cache;
     private readonly IHttpClientFactory _httpFactory;
 
-    public AnimeSearchService(IDataCache cache, IHttpClientFactory httpFactory)
+    public AnimeSearchService(IBotCache cache, IHttpClientFactory httpFactory)
     {
         _cache = cache;
         _httpFactory = httpFactory;
@@ -21,24 +19,25 @@ public class AnimeSearchService : INService
     {
         if (string.IsNullOrWhiteSpace(query))
             throw new ArgumentNullException(nameof(query));
+        
+        TypedKey<AnimeResult> GetKey(string link)
+            => new TypedKey<AnimeResult>($"anime2:{link}");
+        
         try
         {
-            var link = "https://aniapi.nadeko.bot/anime/"
-                       + Uri.EscapeDataString(query.Replace("/", " ", StringComparison.InvariantCulture));
+            var suffix = Uri.EscapeDataString(query.Replace("/", " ", StringComparison.InvariantCulture));
+            var link = $"https://aniapi.nadeko.bot/anime/{suffix}";
             link = link.ToLowerInvariant();
-            var (ok, data) = await _cache.TryGetAnimeDataAsync(link);
-            if (!ok)
+            var result = await _cache.GetAsync(GetKey(link));
+            if (!result.TryPickT0(out var data, out _))
             {
-                using (var http = _httpFactory.CreateClient())
-                {
-                    data = await http.GetStringAsync(link);
-                }
+                using var http = _httpFactory.CreateClient();
+                data = await http.GetFromJsonAsync<AnimeResult>(link);
 
-                await _cache.SetAnimeDataAsync(link, data);
+                await _cache.AddAsync(GetKey(link), data, expiry: TimeSpan.FromHours(12));
             }
 
-
-            return JsonConvert.DeserializeObject<AnimeResult>(data);
+            return data;
         }
         catch
         {
@@ -46,95 +45,31 @@ public class AnimeSearchService : INService
         }
     }
 
-    public async Task<NovelResult> GetNovelData(string query)
-    {
-        if (string.IsNullOrWhiteSpace(query))
-            throw new ArgumentNullException(nameof(query));
-
-        query = query.Replace(" ", "-", StringComparison.InvariantCulture);
-        try
-        {
-            var link = "https://www.novelupdates.com/series/"
-                       + Uri.EscapeDataString(query.Replace(" ", "-").Replace("/", " "));
-            link = link.ToLowerInvariant();
-            var (ok, data) = await _cache.TryGetNovelDataAsync(link);
-            if (!ok)
-            {
-                var config = Configuration.Default.WithDefaultLoader();
-                using var document = await BrowsingContext.New(config).OpenAsync(link);
-                var imageElem = document.QuerySelector("div.seriesimg > img");
-                if (imageElem is null)
-                    return null;
-                var imageUrl = ((IHtmlImageElement)imageElem).Source;
-
-                var descElem = document.QuerySelector("div#editdescription > p");
-                var desc = descElem.InnerHtml;
-
-                var genres = document.QuerySelector("div#seriesgenre")
-                                     .Children.Select(x => x as IHtmlAnchorElement)
-                                     .Where(x => x is not null)
-                                     .Select(x => $"[{x.InnerHtml}]({x.Href})")
-                                     .ToArray();
-
-                var authors = document.QuerySelector("div#showauthors")
-                                      .Children.Select(x => x as IHtmlAnchorElement)
-                                      .Where(x => x is not null)
-                                      .Select(x => $"[{x.InnerHtml}]({x.Href})")
-                                      .ToArray();
-
-                var score = ((IHtmlSpanElement)document.QuerySelector("h5.seriesother > span.uvotes")).InnerHtml;
-
-                var status = document.QuerySelector("div#editstatus").InnerHtml;
-                var title = document.QuerySelector("div.w-blog-content > div.seriestitlenu").InnerHtml;
-
-                var obj = new NovelResult
-                {
-                    Description = desc,
-                    Authors = authors,
-                    Genres = genres,
-                    ImageUrl = imageUrl,
-                    Link = link,
-                    Score = score,
-                    Status = status,
-                    Title = title
-                };
-
-                await _cache.SetNovelDataAsync(link, JsonConvert.SerializeObject(obj));
-
-                return obj;
-            }
-
-            return JsonConvert.DeserializeObject<NovelResult>(data);
-        }
-        catch (Exception ex)
-        {
-            Log.Error(ex, "Error getting novel data");
-            return null;
-        }
-    }
-
     public async Task<MangaResult> GetMangaData(string query)
     {
         if (string.IsNullOrWhiteSpace(query))
             throw new ArgumentNullException(nameof(query));
+        
+        TypedKey<MangaResult> GetKey(string link)
+            => new TypedKey<MangaResult>($"manga2:{link}");
+        
         try
         {
             var link = "https://aniapi.nadeko.bot/manga/"
                        + Uri.EscapeDataString(query.Replace("/", " ", StringComparison.InvariantCulture));
             link = link.ToLowerInvariant();
-            var (ok, data) = await _cache.TryGetAnimeDataAsync(link);
-            if (!ok)
+            
+            var result = await _cache.GetAsync(GetKey(link));
+            if (!result.TryPickT0(out var data, out _))
             {
-                using (var http = _httpFactory.CreateClient())
-                {
-                    data = await http.GetStringAsync(link);
-                }
+                using var http = _httpFactory.CreateClient();
+                data = await http.GetFromJsonAsync<MangaResult>(link);
 
-                await _cache.SetAnimeDataAsync(link, data);
+                await _cache.AddAsync(GetKey(link), data, expiry: TimeSpan.FromHours(3));
             }
 
 
-            return JsonConvert.DeserializeObject<MangaResult>(data);
+            return data;
         }
         catch
         {

src/NadekoBot/Modules/Searches/Anime/MangaResult.cs

@@ -1,32 +1,36 @@
 #nullable disable
-using Newtonsoft.Json;
+using System.Text.Json.Serialization;
 
 namespace NadekoBot.Modules.Searches.Common;
 
 public class MangaResult
 {
+    [JsonPropertyName("id")]
     public int Id { get; set; }
 
-    [JsonProperty("publishing_status")]
+    [JsonPropertyName("publishing_status")]
     public string PublishingStatus { get; set; }
 
-    [JsonProperty("image_url_lge")]
+    [JsonPropertyName("image_url_lge")]
     public string ImageUrlLge { get; set; }
 
-    [JsonProperty("title_english")]
+    [JsonPropertyName("title_english")]
     public string TitleEnglish { get; set; }
 
-    [JsonProperty("total_chapters")]
+    [JsonPropertyName("total_chapters")]
     public int TotalChapters { get; set; }
 
-    [JsonProperty("total_volumes")]
+    [JsonPropertyName("total_volumes")]
     public int TotalVolumes { get; set; }
 
+    [JsonPropertyName("description")]
     public string Description { get; set; }
+    
+    [JsonPropertyName("genres")]
     public string[] Genres { get; set; }
 
-    [JsonProperty("average_score")]
-    public string AverageScore { get; set; }
+    [JsonPropertyName("average_score")]
+    public float AverageScore { get; set; }
 
     public string Link
         => "http://anilist.co/manga/" + Id;

src/NadekoBot/Modules/Searches/Crypto/CryptoService.cs

@@ -15,13 +15,13 @@ namespace NadekoBot.Modules.Searches.Services;
 
 public class CryptoService : INService
 {
-    private readonly IDataCache _cache;
+    private readonly IBotCache _cache;
     private readonly IHttpClientFactory _httpFactory;
     private readonly IBotCredentials _creds;
 
     private readonly SemaphoreSlim _getCryptoLock = new(1, 1);
 
-    public CryptoService(IDataCache cache, IHttpClientFactory httpFactory, IBotCredentials creds)
+    public CryptoService(IBotCache cache, IHttpClientFactory httpFactory, IBotCredentials creds)
     {
         _cache = cache;
         _httpFactory = httpFactory;
@@ -40,7 +40,8 @@ public class CryptoService : INService
         Span<PointF> points = new PointF[gElement.ChildNodes.Count];
         var cnt = 0;
 
-        bool GetValuesFromAttributes(XmlAttributeCollection attrs,
+        bool GetValuesFromAttributes(
+            XmlAttributeCollection attrs,
             out float x1,
             out float y1,
             out float x2,
@@ -56,7 +57,7 @@ public class CryptoService : INService
                    && attrs["y2"]?.Value is string y2Str
                    && float.TryParse(y2Str, NumberStyles.Any, CultureInfo.InvariantCulture, out y2);
         }
-        
+
         foreach (XmlElement x in gElement.ChildNodes)
         {
             if (x.Name != "line")
@@ -67,22 +68,22 @@ public class CryptoService : INService
                 points[cnt++] = new(x1, y1);
                 // this point will be set twice to the same value
                 // on all points except the last one
-                if(cnt + 1 < points.Length)
+                if (cnt + 1 < points.Length)
                     points[cnt + 1] = new(x2, y2);
             }
         }
 
         if (cnt == 0)
             return Array.Empty<PointF>();
-        
+
         return points.Slice(0, cnt).ToArray();
     }
-    
+
     private SixLabors.ImageSharp.Image<Rgba32> GenerateSparklineChart(PointF[] points, bool up)
     {
         const int width = 164;
         const int height = 48;
-        
+
         var img = new Image<Rgba32>(width, height, Color.Transparent);
         var color = up
             ? Color.Green
@@ -92,10 +93,10 @@ public class CryptoService : INService
         {
             x.DrawLines(color, 2, points);
         });
-        
+
         return img;
     }
-    
+
     public async Task<(CmcResponseData? Data, CmcResponseData? Nearest)> GetCryptoData(string name)
     {
         if (string.IsNullOrWhiteSpace(name))
@@ -130,20 +131,20 @@ public class CryptoService : INService
         await _getCryptoLock.WaitAsync();
         try
         {
-            var fullStrData = await _cache.GetOrAddCachedDataAsync("nadeko:crypto_data",
-                async _ =>
+            var data = await _cache.GetOrAddAsync(new("nadeko:crypto_data"),
+                async () =>
                 {
                     try
                     {
                         using var http = _httpFactory.CreateClient();
-                        var strData = await http.GetFromJsonAsync<CryptoResponse>(
+                        var data = await http.GetFromJsonAsync<CryptoResponse>(
                             "https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest?"
                             + $"CMC_PRO_API_KEY={_creds.CoinmarketcapApiKey}"
                             + "&start=1"
                             + "&limit=5000"
                             + "&convert=USD");
 
-                        return JsonSerializer.Serialize(strData);
+                        return data;
                     }
                     catch (Exception ex)
                     {
@@ -151,13 +152,12 @@ public class CryptoService : INService
                         return default;
                     }
                 },
-                "",
                 TimeSpan.FromHours(2));
 
-            if (fullStrData is null)
+            if (data is null)
                 return default;
-            
-            return JsonSerializer.Deserialize<CryptoResponse>(fullStrData)?.Data ?? new();
+
+            return data.Data;
         }
         catch (Exception ex)
         {
@@ -170,44 +170,33 @@ public class CryptoService : INService
         }
     }
 
+    private TypedKey<byte[]> GetSparklineKey(int id)
+        => new($"crypto:sparkline:{id}");
+
     public async Task<Stream?> GetSparklineAsync(int id, bool up)
-    {
-        var key = $"crypto:sparkline:{id}";
-        
-        // attempt to get from cache
-        var db = _cache.Redis.GetDatabase();
-        byte[] bytes = await db.StringGetAsync(key);
-        // if it succeeds, return it
-        if (bytes is { Length: > 0 })
-        {
-            return bytes.ToStream();
-        }
-        
-        // if it fails, generate a new one
-        var points = await DownloadSparklinePointsAsync(id);
-        if (points is null)
-            return default;
-        
-        var sparkline = GenerateSparklineChart(points, up);
-
-        // add to cache for 1h and return it
-        
-        var stream = sparkline.ToStream();
-        await db.StringSetAsync(key, stream.ToArray(), expiry: TimeSpan.FromHours(1));
-        return stream;
-    }
-
-    private async Task<PointF[]?> DownloadSparklinePointsAsync(int id)
     {
         try
         {
-            using var http = _httpFactory.CreateClient();
-            var str = await http.GetStringAsync(
-                $"https://s3.coinmarketcap.com/generated/sparklines/web/7d/usd/{id}.svg");
-            var points = GetSparklinePointsFromSvgText(str);
-            return points;
+            var bytes = await _cache.GetOrAddAsync(GetSparklineKey(id),
+                async () =>
+                {
+                    // if it fails, generate a new one
+                    var points = await DownloadSparklinePointsAsync(id);
+                    var sparkline = GenerateSparklineChart(points, up);
+
+                    using var stream = await sparkline.ToStreamAsync();
+                    return stream.ToArray();
+                },
+                TimeSpan.FromHours(1));
+
+            if (bytes is { Length: > 0 })
+            {
+                return bytes.ToStream();
+            }
+
+            return default;
         }
-        catch(Exception ex)
+        catch (Exception ex)
         {
             Log.Warning(ex,
                 "Exception occurred while downloading sparkline points: {ErrorMessage}",
@@ -215,4 +204,13 @@ public class CryptoService : INService
             return default;
         }
     }
+
+    private async Task<PointF[]> DownloadSparklinePointsAsync(int id)
+    {
+        using var http = _httpFactory.CreateClient();
+        var str = await http.GetStringAsync(
+            $"https://s3.coinmarketcap.com/generated/sparklines/web/7d/usd/{id}.svg");
+        var points = GetSparklinePointsFromSvgText(str);
+        return points;
+    }
 }

src/NadekoBot/Modules/Searches/Feeds/FeedsService.cs

@@ -49,7 +49,7 @@ public class FeedsService : INService
     private void ClearErrors(string url)
         => _errorCounters.Remove(url);
 
-    private async Task AddError(string url, List<int> ids)
+    private async Task<uint> AddError(string url, List<int> ids)
     {
         try
         {
@@ -68,10 +68,13 @@ public class FeedsService : INService
                 // reset the error counter
                 ClearErrors(url);
             }
+
+            return newValue;
         }
         catch (Exception ex)
         {
             Log.Error(ex, "Error adding rss errors...");
+            return 0;
         }
     }
 
@@ -181,12 +184,13 @@ public class FeedsService : INService
                 }
                 catch (Exception ex)
                 {
-                    Log.Warning("An error occured while getting rss stream {RssFeed}"
+                    var errorCount = await AddError(rssUrl, kvp.Value.Select(x => x.Id).ToList());
+                    
+                    Log.Warning("An error occured while getting rss stream ({ErrorCount} / 100) {RssFeed}"
                                 + "\n {Message}",
+                        errorCount,
                         rssUrl,
                         $"[{ex.GetType().Name}]: {ex.Message}");
-
-                    await AddError(rssUrl, kvp.Value.Select(x => x.Id).ToList());
                 }
             }
 

src/NadekoBot/Modules/Searches/PokemonSearchCommands.cs

@@ -9,15 +9,9 @@ public partial class Searches
     [Group]
     public partial class PokemonSearchCommands : NadekoModule<SearchesService>
     {
-        public IReadOnlyDictionary<string, SearchPokemon> Pokemons
-            => _cache.LocalData.Pokemons;
+        private readonly ILocalDataCache _cache;
 
-        public IReadOnlyDictionary<string, SearchPokemonAbility> PokemonAbilities
-            => _cache.LocalData.PokemonAbilities;
-
-        private readonly IDataCache _cache;
-
-        public PokemonSearchCommands(IDataCache cache)
+        public PokemonSearchCommands(ILocalDataCache cache)
             => _cache = cache;
 
         [Cmd]
@@ -27,7 +21,7 @@ public partial class Searches
             if (string.IsNullOrWhiteSpace(pokemon))
                 return;
 
-            foreach (var kvp in Pokemons)
+            foreach (var kvp in await _cache.GetPokemonsAsync())
             {
                 if (kvp.Key.ToUpperInvariant() == pokemon.ToUpperInvariant())
                 {
@@ -58,7 +52,7 @@ public partial class Searches
             ability = ability?.Trim().ToUpperInvariant().Replace(" ", "", StringComparison.InvariantCulture);
             if (string.IsNullOrWhiteSpace(ability))
                 return;
-            foreach (var kvp in PokemonAbilities)
+            foreach (var kvp in await _cache.GetPokemonAbilitiesAsync())
             {
                 if (kvp.Key.ToUpperInvariant() == ability)
                 {

src/NadekoBot/Modules/Searches/Search/SearchCommands.cs

@@ -10,14 +10,14 @@ public partial class Searches
     public partial class SearchCommands : NadekoModule
     {
         private readonly ISearchServiceFactory _searchFactory;
-        private readonly ConnectionMultiplexer _redis;
+        private readonly IBotCache _cache;
 
         public SearchCommands(
             ISearchServiceFactory searchFactory,
-            ConnectionMultiplexer redis)
+            IBotCache cache)
         {
             _searchFactory = searchFactory;
-            _redis = redis;
+            _cache = cache;
         }
 
         [Cmd]
@@ -127,18 +127,17 @@ public partial class Searches
             await ctx.Channel.EmbedAsync(null, embeds: embeds);
         }
 
+        private TypedKey<string> GetYtCacheKey(string query)
+            => new($"search:youtube:{query}");
+        
         private async Task AddYoutubeUrlToCacheAsync(string query, string url)
-        {
-            var db = _redis.GetDatabase();
-            await db.StringSetAsync($"search:youtube:{query}", url, expiry: 1.Hours());
-        }
+            => await _cache.AddAsync(GetYtCacheKey(query), url, expiry: 1.Hours());
 
         private async Task<VideoInfo?> GetYoutubeUrlFromCacheAsync(string query)
         {
-            var db = _redis.GetDatabase();
-            var url = await db.StringGetAsync($"search:youtube:{query}");
+            var result = await _cache.GetAsync(GetYtCacheKey(query));
 
-            if (string.IsNullOrWhiteSpace(url))
+            if (!result.TryGetValue(out var url) || string.IsNullOrWhiteSpace(url))
                 return null;
 
             return new VideoInfo()

src/NadekoBot/Modules/Searches/SearchesService.cs

@@ -1,6 +1,4 @@
 #nullable disable
-using AngleSharp.Html.Dom;
-using AngleSharp.Html.Parser;
 using Html2Markdown;
 using NadekoBot.Modules.Searches.Common;
 using Newtonsoft.Json;
@@ -10,7 +8,6 @@ using SixLabors.ImageSharp;
 using SixLabors.ImageSharp.Drawing.Processing;
 using SixLabors.ImageSharp.PixelFormats;
 using SixLabors.ImageSharp.Processing;
-using System.Net;
 using Color = SixLabors.ImageSharp.Color;
 using Image = SixLabors.ImageSharp.Image;
 
@@ -31,9 +28,9 @@ public class SearchesService : INService
     private readonly IHttpClientFactory _httpFactory;
     private readonly IGoogleApiService _google;
     private readonly IImageCache _imgs;
-    private readonly IDataCache _cache;
+    private readonly IBotCache _c;
     private readonly FontProvider _fonts;
-    private readonly IBotCredentials _creds;
+    private readonly IBotCredsProvider _creds;
     private readonly NadekoRandom _rng;
     private readonly List<string> _yomamaJokes;
 
@@ -42,15 +39,16 @@ public class SearchesService : INService
 
     public SearchesService(
         IGoogleApiService google,
-        IDataCache cache,
+        IImageCache images,
+        IBotCache c,
         IHttpClientFactory factory,
         FontProvider fonts,
-        IBotCredentials creds)
+        IBotCredsProvider creds)
     {
         _httpFactory = factory;
         _google = google;
-        _imgs = cache.LocalImages;
-        _cache = cache;
+        _imgs = images;
+        _c = c;
         _fonts = fonts;
         _creds = creds;
         _rng = new();
@@ -76,36 +74,28 @@ public class SearchesService : INService
     }
 
     public async Task<Stream> GetRipPictureAsync(string text, Uri imgUrl)
-    {
-        var data = await _cache.GetOrAddCachedDataAsync($"nadeko_rip_{text}_{imgUrl}",
-            GetRipPictureFactory,
-            (text, imgUrl),
-            TimeSpan.FromDays(1));
-
-        return data.ToStream();
-    }
+        => (await GetRipPictureFactory(text, imgUrl)).ToStream();
 
     private void DrawAvatar(Image bg, Image avatarImage)
         => bg.Mutate(x => x.Grayscale().DrawImage(avatarImage, new(83, 139), new GraphicsOptions()));
 
-    public async Task<byte[]> GetRipPictureFactory((string text, Uri avatarUrl) arg)
+    public async Task<byte[]> GetRipPictureFactory(string text, Uri avatarUrl)
     {
-        var (text, avatarUrl) = arg;
-        using var bg = Image.Load<Rgba32>(_imgs.Rip.ToArray());
-        var (succ, data) = (false, (byte[])null); //await _cache.TryGetImageDataAsync(avatarUrl);
-        if (!succ)
+        using var bg = Image.Load<Rgba32>(await _imgs.GetRipBgAsync());
+        var result = await _c.GetImageDataAsync(avatarUrl);
+        if (!result.TryPickT0(out var data, out _))
         {
             using var http = _httpFactory.CreateClient();
             data = await http.GetByteArrayAsync(avatarUrl);
             using (var avatarImg = Image.Load<Rgba32>(data))
             {
                 avatarImg.Mutate(x => x.Resize(85, 85).ApplyRoundedCorners(42));
-                await using var avStream = avatarImg.ToStream();
+                await using var avStream = await avatarImg.ToStreamAsync();
                 data = avStream.ToArray();
                 DrawAvatar(bg, avatarImg);
             }
 
-            await _cache.SetImageDataAsync(avatarUrl, data);
+            await _c.SetImageDataAsync(avatarUrl, data);
         }
         else
         {
@@ -128,7 +118,7 @@ public class SearchesService : INService
             new(25, 225)));
 
         //flowa
-        using (var flowers = Image.Load(_imgs.RipOverlay.ToArray()))
+        using (var flowers = Image.Load(await _imgs.GetRipOverlayAsync()))
         {
             bg.Mutate(x => x.DrawImage(flowers, new(0, 0), new GraphicsOptions()));
         }
@@ -137,13 +127,12 @@ public class SearchesService : INService
         return stream.ToArray();
     }
 
-    public Task<WeatherData> GetWeatherDataAsync(string query)
+    public async Task<WeatherData> GetWeatherDataAsync(string query)
     {
         query = query.Trim().ToLowerInvariant();
 
-        return _cache.GetOrAddCachedDataAsync($"nadeko_weather_{query}",
-            GetWeatherDataFactory,
-            query,
+        return await _c.GetOrAddAsync(new($"nadeko_weather_{query}"),
+            async () => await GetWeatherDataFactory(query),
             TimeSpan.FromHours(3));
     }
 
@@ -184,26 +173,28 @@ public class SearchesService : INService
         if (string.IsNullOrEmpty(query))
             return (default, TimeErrors.InvalidInput);
 
-        if (string.IsNullOrWhiteSpace(_creds.LocationIqApiKey) || string.IsNullOrWhiteSpace(_creds.TimezoneDbApiKey))
+
+        var locIqKey = _creds.GetCreds().LocationIqApiKey;
+        var tzDbKey = _creds.GetCreds().TimezoneDbApiKey;
+        if (string.IsNullOrWhiteSpace(locIqKey) || string.IsNullOrWhiteSpace(tzDbKey))
             return (default, TimeErrors.ApiKeyMissing);
 
         try
         {
             using var http = _httpFactory.CreateClient();
-            var res = await _cache.GetOrAddCachedDataAsync($"geo_{query}",
-                _ =>
+            var res = await _c.GetOrAddAsync(new($"searches:geo:{query}"),
+                async () =>
                 {
                     var url = "https://eu1.locationiq.com/v1/search.php?"
-                              + (string.IsNullOrWhiteSpace(_creds.LocationIqApiKey)
+                              + (string.IsNullOrWhiteSpace(locIqKey)
                                   ? "key="
-                                  : $"key={_creds.LocationIqApiKey}&")
+                                  : $"key={locIqKey}&")
                               + $"q={Uri.EscapeDataString(query)}&"
                               + "format=json";
 
-                    var res = http.GetStringAsync(url);
+                    var res = await http.GetStringAsync(url);
                     return res;
                 },
-                "",
                 TimeSpan.FromHours(1));
 
             var responses = JsonConvert.DeserializeObject<LocationIqResponse[]>(res);
@@ -217,7 +208,7 @@ public class SearchesService : INService
 
             using var req = new HttpRequestMessage(HttpMethod.Get,
                 "http://api.timezonedb.com/v2.1/get-time-zone?"
-                + $"key={_creds.TimezoneDbApiKey}"
+                + $"key={tzDbKey}"
                 + $"&format=json"
                 + $"&by=position"
                 + $"&lat={geoData.Lat}"
@@ -315,9 +306,8 @@ public class SearchesService : INService
     public async Task<MtgData> GetMtgCardAsync(string search)
     {
         search = search.Trim().ToLowerInvariant();
-        var data = await _cache.GetOrAddCachedDataAsync($"nadeko_mtg_{search}",
-            GetMtgCardFactory,
-            search,
+        var data = await _c.GetOrAddAsync(new($"mtg:{search}"),
+            async () => await GetMtgCardFactory(search),
             TimeSpan.FromDays(1));
 
         if (data is null || data.Length == 0)
@@ -368,12 +358,11 @@ public class SearchesService : INService
         return await cards.Select(GetMtgDataAsync).WhenAll();
     }
 
-    public Task<HearthstoneCardData> GetHearthstoneCardDataAsync(string name)
+    public async Task<HearthstoneCardData> GetHearthstoneCardDataAsync(string name)
     {
         name = name.ToLowerInvariant();
-        return _cache.GetOrAddCachedDataAsync($"nadeko_hearthstone_{name}",
-            HearthstoneCardDataFactory,
-            name,
+        return await _c.GetOrAddAsync($"hearthstone:{name}",
+            () => HearthstoneCardDataFactory(name),
             TimeSpan.FromDays(1));
     }
 
@@ -381,7 +370,7 @@ public class SearchesService : INService
     {
         using var http = _httpFactory.CreateClient();
         http.DefaultRequestHeaders.Clear();
-        http.DefaultRequestHeaders.Add("x-rapidapi-key", _creds.RapidApiKey);
+        http.DefaultRequestHeaders.Add("x-rapidapi-key", _creds.GetCreds().RapidApiKey);
         try
         {
             var response = await http.GetStringAsync("https://omgvamp-hearthstone-v1.p.rapidapi.com/"
@@ -410,16 +399,22 @@ public class SearchesService : INService
         }
     }
 
-    public Task<OmdbMovie> GetMovieDataAsync(string name)
+    public async Task<OmdbMovie> GetMovieDataAsync(string name)
     {
         name = name.Trim().ToLowerInvariant();
-        return _cache.GetOrAddCachedDataAsync($"nadeko_movie_{name}", GetMovieDataFactory, name, TimeSpan.FromDays(1));
+        return await _c.GetOrAddAsync(new($"movie:{name}"),
+            () => GetMovieDataFactory(name),
+            TimeSpan.FromDays(1));
     }
 
     private async Task<OmdbMovie> GetMovieDataFactory(string name)
     {
         using var http = _httpFactory.CreateClient();
-        var res = await http.GetStringAsync(string.Format("https://omdbapi.nadeko.bot/?t={0}&y=&plot=full&r=json",
+        var res = await http.GetStringAsync(string.Format("https://omdbapi.nadeko.bot/"
+                                                          + "?t={0}"
+                                                          + "&y="
+                                                          + "&plot=full"
+                                                          + "&r=json",
             name.Trim().Replace(' ', '+')));
         var movie = JsonConvert.DeserializeObject<OmdbMovie>(res);
         if (movie?.Title is null)
@@ -432,10 +427,11 @@ public class SearchesService : INService
     {
         const string steamGameIdsKey = "steam_names_to_appid";
 
-        var gamesMap = await _cache.GetOrAddCachedDataAsync(steamGameIdsKey,
-            async _ =>
+        var gamesMap = await _c.GetOrAddAsync(new(steamGameIdsKey),
+            async () =>
             {
                 using var http = _httpFactory.CreateClient();
+                
                 // https://api.steampowered.com/ISteamApps/GetAppList/v2/
                 var gamesStr = await http.GetStringAsync("https://api.steampowered.com/ISteamApps/GetAppList/v2/");
                 var apps = JsonConvert
@@ -446,23 +442,18 @@ public class SearchesService : INService
                                    {
                                        apps = new List<SteamGameId>()
                                    }
-                               })
+                               })!
                            .applist.apps;
 
                 return apps.OrderBy(x => x.Name, StringComparer.OrdinalIgnoreCase)
                            .GroupBy(x => x.Name)
                            .ToDictionary(x => x.Key, x => x.First().AppId);
-                //await db.HashSetAsync("steam_game_ids", apps.Select(app => new HashEntry(app.Name.Trim().ToLowerInvariant(), app.AppId)).ToArray());
-                //await db.StringSetAsync("steam_game_ids", gamesStr, TimeSpan.FromHours(24));
-                //await db.KeyExpireAsync("steam_game_ids", TimeSpan.FromHours(24), CommandFlags.FireAndForget);
             },
-            default(string),
             TimeSpan.FromHours(24));
 
         if (gamesMap is null)
             return -1;
-
-
+        
         query = query.Trim();
 
         var keyList = gamesMap.Keys.ToList();

src/NadekoBot/Modules/Searches/StreamNotification/StreamNotificationService.cs

@@ -1,4 +1,6 @@
 #nullable disable
+using LinqToDB;
+using LinqToDB.EntityFrameworkCore;
 using Microsoft.EntityFrameworkCore;
 using NadekoBot.Common.ModuleBehaviors;
 using NadekoBot.Db;
@@ -6,10 +8,98 @@ using NadekoBot.Db.Models;
 using NadekoBot.Modules.Searches.Common;
 using NadekoBot.Modules.Searches.Common.StreamNotifications;
 using NadekoBot.Services.Database.Models;
-using StackExchange.Redis;
 
 namespace NadekoBot.Modules.Searches.Services;
 
+public sealed class StreamOnlineMessageDeleterService : INService, IReadyExecutor
+{
+    private readonly StreamNotificationService _notifService;
+    private readonly DbService _db;
+    private readonly DiscordSocketClient _client;
+    private readonly IPubSub _pubSub;
+
+    public StreamOnlineMessageDeleterService(
+        StreamNotificationService notifService,
+        DbService db,
+        IPubSub pubSub,
+        DiscordSocketClient client)
+    {
+        _notifService = notifService;
+        _db = db;
+        _client = client;
+        _pubSub = pubSub;
+    }
+
+    public async Task OnReadyAsync()
+    {
+        _notifService.OnlineMessagesSent += OnOnlineMessagesSent;
+        
+        if(_client.ShardId == 0)
+            await _pubSub.Sub(_notifService.StreamsOfflineKey, OnStreamsOffline);
+    }
+
+    private async Task OnOnlineMessagesSent(FollowedStream.FType type, string name, IReadOnlyCollection<(ulong, ulong)> pairs)
+    {
+        await using var ctx = _db.GetDbContext();
+        foreach (var (channelId, messageId) in pairs)
+        {
+            await ctx.GetTable<StreamOnlineMessage>()
+                     .InsertAsync(() => new()
+                     {
+                         Name = name,
+                         Type = type,
+                         MessageId = messageId,
+                         ChannelId = channelId,
+                         DateAdded = DateTime.UtcNow,
+                     });
+        }
+    }
+
+    private async ValueTask OnStreamsOffline(List<StreamData> streamDatas)
+    {
+        if (_client.ShardId != 0)
+            return;
+
+        var pairs = await GetMessagesToDelete(streamDatas);
+
+        foreach (var (channelId, messageId) in pairs)
+        {
+            try
+            {
+                var textChannel = await _client.GetChannelAsync(channelId) as ITextChannel;
+                if (textChannel is null)
+                    continue;
+
+                await textChannel.DeleteMessageAsync(messageId);
+            }
+            catch
+            {
+                continue;
+            }
+        }
+    }
+
+    private async Task<IEnumerable<(ulong, ulong)>> GetMessagesToDelete(List<StreamData> streamDatas)
+    {
+        await using var ctx = _db.GetDbContext();
+
+        var toReturn = new List<(ulong, ulong)>();
+        foreach (var sd in streamDatas)
+        {
+            var key = sd.CreateKey();
+            var toDelete = await ctx.GetTable<StreamOnlineMessage>()
+                                    .Where(x => (x.Type == key.Type && x.Name == key.Name)
+                                                || Sql.DateDiff(Sql.DateParts.Day, x.DateAdded, DateTime.UtcNow) > 1)
+                                    .DeleteWithOutputAsync();
+            
+            toReturn.AddRange(toDelete.Select(x => (x.ChannelId, x.MessageId)));
+        }
+
+        return toReturn;
+    }
+}
+
+
 public sealed class StreamNotificationService : INService, IReadyExecutor
 {
     private readonly DbService _db;
@@ -29,18 +119,22 @@ public sealed class StreamNotificationService : INService, IReadyExecutor
     private readonly IPubSub _pubSub;
     private readonly IEmbedBuilderService _eb;
 
-    private readonly TypedKey<List<StreamData>> _streamsOnlineKey;
-    private readonly TypedKey<List<StreamData>> _streamsOfflineKey;
+    public TypedKey<List<StreamData>> StreamsOnlineKey { get; }
+    public TypedKey<List<StreamData>> StreamsOfflineKey { get; }
 
     private readonly TypedKey<FollowStreamPubData> _streamFollowKey;
     private readonly TypedKey<FollowStreamPubData> _streamUnfollowKey;
-    private readonly ConnectionMultiplexer _redis;
+
+    public event Func<
+        FollowedStream.FType,
+        string,
+        IReadOnlyCollection<(ulong, ulong)>,
+        Task> OnlineMessagesSent = static delegate { return Task.CompletedTask; };
 
     public StreamNotificationService(
         DbService db,
         DiscordSocketClient client,
         IBotStrings strings,
-        ConnectionMultiplexer redis,
         IBotCredsProvider creds,
         IHttpClientFactory httpFactory,
         Bot bot,
@@ -52,11 +146,11 @@ public sealed class StreamNotificationService : INService, IReadyExecutor
         _strings = strings;
         _pubSub = pubSub;
         _eb = eb;
-        _redis = redis;
-        _streamTracker = new(httpFactory, creds, redis, creds.GetCreds().RedisKey(), client.ShardId == 0);
+        
+        _streamTracker = new(httpFactory, creds);
 
-        _streamsOnlineKey = new("streams.online");
-        _streamsOfflineKey = new("streams.offline");
+        StreamsOnlineKey = new("streams.online");
+        StreamsOfflineKey = new("streams.offline");
 
         _streamFollowKey = new("stream.follow");
         _streamUnfollowKey = new("stream.unfollow");
@@ -100,7 +194,7 @@ public sealed class StreamNotificationService : INService, IReadyExecutor
                 var allFollowedStreams = uow.Set<FollowedStream>().AsQueryable().ToList();
 
                 foreach (var fs in allFollowedStreams)
-                    _streamTracker.CacheAddData(fs.CreateKey(), null, false);
+                    _streamTracker.AddLastData(fs.CreateKey(), null, false);
 
                 _trackCounter = allFollowedStreams.GroupBy(x => new
                                                   {
@@ -112,8 +206,8 @@ public sealed class StreamNotificationService : INService, IReadyExecutor
             }
         }
 
-        _pubSub.Sub(_streamsOfflineKey, HandleStreamsOffline);
-        _pubSub.Sub(_streamsOnlineKey, HandleStreamsOnline);
+        _pubSub.Sub(StreamsOfflineKey, HandleStreamsOffline);
+        _pubSub.Sub(StreamsOnlineKey, HandleStreamsOnline);
 
         if (client.ShardId == 0)
         {
@@ -186,7 +280,7 @@ public sealed class StreamNotificationService : INService, IReadyExecutor
     /// </summary>
     private ValueTask HandleFollowStream(FollowStreamPubData info)
     {
-        _streamTracker.CacheAddData(info.Key, null, false);
+        _streamTracker.AddLastData(info.Key, null, false);
         lock (_shardLock)
         {
             var key = info.Key;
@@ -251,45 +345,8 @@ public sealed class StreamNotificationService : INService, IReadyExecutor
                       .WhenAll();
             }
         }
-
-        if (_client.ShardId == 0)
-        {
-            foreach (var stream in offlineStreams)
-            {
-                await DeleteOnlineMessages(stream);
-            }
-        }
-    }
-
-    private async Task DeleteOnlineMessages(StreamData stream)
-    {
-        var db = _redis.GetDatabase();
-        var data = await db.ListRangeAsync($"streams_online_del:{stream.CreateKey()}");
-        await db.KeyDeleteAsync($"streams_online_del:{stream.CreateKey()}");
-        
-        foreach (string pair in data)
-        {
-            var pairArr = pair.Split(',');
-            if (pairArr.Length != 2)
-                continue;
-
-            if (!ulong.TryParse(pairArr[0], out var chId) || !ulong.TryParse(pairArr[1], out var msgId))
-                continue;
-
-            try
-            {
-                var textChannel = await _client.GetChannelAsync(chId) as ITextChannel;
-                if (textChannel is null)
-                    continue;
-
-                await textChannel.DeleteMessageAsync(msgId);
-            }
-            catch
-            {
-                continue;
-            }
-        }
     }
+    
 
     private async ValueTask HandleStreamsOnline(List<StreamData> onlineStreams)
     {
@@ -331,14 +388,11 @@ public sealed class StreamNotificationService : INService, IReadyExecutor
                 {
                     var pairs = messages
                                 .Where(x => x != default)
-                                .Select(x => (RedisValue)$"{x.Item1},{x.Item2}")
-                                .ToArray();
+                                .Select(x => (x.Item1, x.Item2))
+                                .ToList();
 
-                    if (pairs.Length > 0)
-                    {
-                        var db = _redis.GetDatabase();
-                        await db.ListRightPushAsync($"streams_online_del:{key}", pairs);
-                    }
+                    if (pairs.Count > 0)
+                        await OnlineMessagesSent(key.Type, key.Name, pairs);
                 }
                 catch
                 {
@@ -349,10 +403,10 @@ public sealed class StreamNotificationService : INService, IReadyExecutor
     }
 
     private Task OnStreamsOnline(List<StreamData> data)
-        => _pubSub.Pub(_streamsOnlineKey, data);
+        => _pubSub.Pub(StreamsOnlineKey, data);
 
     private Task OnStreamsOffline(List<StreamData> data)
-        => _pubSub.Pub(_streamsOfflineKey, data);
+        => _pubSub.Pub(StreamsOfflineKey, data);
 
     private Task ClientOnJoinedGuild(GuildConfig guildConfig)
     {

src/NadekoBot/Modules/Searches/_Common/CryptoData.cs

@@ -16,11 +16,11 @@ public class CmcQuote
     [JsonPropertyName("volume_24h")]
     public double Volume24h { get; set; }
 
-    [JsonPropertyName("volume_change_24h")]
-    public double VolumeChange24h { get; set; }
-
-    [JsonPropertyName("percent_change_1h")]
-    public double PercentChange1h { get; set; }
+    // [JsonPropertyName("volume_change_24h")]
+    // public double VolumeChange24h { get; set; }
+    //
+    // [JsonPropertyName("percent_change_1h")]
+    // public double PercentChange1h { get; set; }
 
     [JsonPropertyName("percent_change_24h")]
     public double PercentChange24h { get; set; }
@@ -33,12 +33,6 @@ public class CmcQuote
 
     [JsonPropertyName("market_cap_dominance")]
     public double MarketCapDominance { get; set; }
-
-    [JsonPropertyName("fully_diluted_market_cap")]
-    public double FullyDilutedMarketCap { get; set; }
-
-    [JsonPropertyName("last_updated")]
-    public DateTime LastUpdated { get; set; }
 }
 
 public class CmcResponseData
@@ -58,9 +52,6 @@ public class CmcResponseData
     [JsonPropertyName("cmc_rank")]
     public int CmcRank { get; set; }
 
-    [JsonPropertyName("num_market_pairs")]
-    public int NumMarketPairs { get; set; }
-
     [JsonPropertyName("circulating_supply")]
     public double? CirculatingSupply { get; set; }
 
@@ -70,15 +61,6 @@ public class CmcResponseData
     [JsonPropertyName("max_supply")]
     public double? MaxSupply { get; set; }
 
-    [JsonPropertyName("last_updated")]
-    public DateTime LastUpdated { get; set; }
-
-    [JsonPropertyName("date_added")]
-    public DateTime DateAdded { get; set; }
-
-    [JsonPropertyName("tags")]
-    public List<string> Tags { get; set; }
-
     [JsonPropertyName("quote")]
     public Dictionary<string, CmcQuote> Quote { get; set; }
 }

src/NadekoBot/Modules/Searches/_Common/StreamNotifications/Models/StreamDataKey.cs

@@ -6,10 +6,9 @@ namespace NadekoBot.Modules.Searches.Common;
 
 public readonly struct StreamDataKey
 {
-    public FollowedStream.FType Type { get; }
-    public string Name { get; }
+    public FollowedStream.FType Type { get; init; }
+    public string Name { get; init; }
 
-    [JsonConstructor]
     public StreamDataKey(FollowedStream.FType type, string name)
     {
         Type = type;

src/NadekoBot/Modules/Searches/_Common/StreamNotifications/NotifChecker.cs

@@ -1,7 +1,5 @@
 using NadekoBot.Db.Models;
 using NadekoBot.Modules.Searches.Common.StreamNotifications.Providers;
-using Newtonsoft.Json;
-using StackExchange.Redis;
 
 namespace NadekoBot.Modules.Searches.Common.StreamNotifications;
 
@@ -9,30 +7,22 @@ public class NotifChecker
 {
     public event Func<List<StreamData>, Task> OnStreamsOffline = _ => Task.CompletedTask;
     public event Func<List<StreamData>, Task> OnStreamsOnline = _ => Task.CompletedTask;
-    private readonly ConnectionMultiplexer _multi;
-    private readonly string _key;
 
-    private readonly Dictionary<FollowedStream.FType, Provider> _streamProviders;
+    private readonly IReadOnlyDictionary<FollowedStream.FType, Provider> _streamProviders;
     private readonly HashSet<(FollowedStream.FType, string)> _offlineBuffer;
+    private readonly ConcurrentDictionary<StreamDataKey, StreamData?> _cache = new();
 
     public NotifChecker(
         IHttpClientFactory httpClientFactory,
-        IBotCredsProvider credsProvider,
-        ConnectionMultiplexer multi,
-        string uniqueCacheKey,
-        bool isMaster)
+        IBotCredsProvider credsProvider)
     {
-        _multi = multi;
-        _key = $"{uniqueCacheKey}_followed_streams_data";
-        _streamProviders = new()
+        _streamProviders = new Dictionary<FollowedStream.FType, Provider>()
         {
             { FollowedStream.FType.Twitch, new TwitchHelixProvider(httpClientFactory, credsProvider) },
             { FollowedStream.FType.Picarto, new PicartoProvider(httpClientFactory) },
             { FollowedStream.FType.Trovo, new TrovoProvider(httpClientFactory, credsProvider) }
         };
         _offlineBuffer = new();
-        if (isMaster)
-            CacheClearAllData();
     }
 
     // gets all streams which have been failing for more than the provided timespan
@@ -61,7 +51,7 @@ public class NotifChecker
             {
                 try
                 {
-                    var allStreamData = CacheGetAllData();
+                    var allStreamData = GetAllData();
 
                     var oldStreamDataDict = allStreamData
                                             // group by type
@@ -101,7 +91,7 @@ public class NotifChecker
                             || !typeDict.TryGetValue(key.Name, out var oldData)
                             || oldData is null)
                         {
-                            CacheAddData(key, newData, true);
+                            AddLastData(key, newData, true);
                             continue;
                         }
                         
@@ -109,7 +99,7 @@ public class NotifChecker
                         if (string.IsNullOrWhiteSpace(newData.Game))
                             newData.Game = oldData.Game;
                         
-                        CacheAddData(key, newData, true);
+                        AddLastData(key, newData, true);
 
                         // if the stream is offline, we need to check if it was
                         // marked as offline once previously
@@ -158,39 +148,22 @@ public class NotifChecker
             }
         });
 
-    public bool CacheAddData(StreamDataKey key, StreamData? data, bool replace)
+    public bool AddLastData(StreamDataKey key, StreamData? data, bool replace)
     {
-        var db = _multi.GetDatabase();
-        return db.HashSet(_key,
-            JsonConvert.SerializeObject(key),
-            JsonConvert.SerializeObject(data),
-            replace ? When.Always : When.NotExists);
+        if (replace)
+        {
+            _cache[key] = data;
+            return true;
+        }
+        
+        return _cache.TryAdd(key, data);
     }
 
-    public void CacheDeleteData(StreamDataKey key)
-    {
-        var db = _multi.GetDatabase();
-        db.HashDelete(_key, JsonConvert.SerializeObject(key));
-    }
+    public void DeleteLastData(StreamDataKey key)
+        => _cache.TryRemove(key, out _);
 
-    public void CacheClearAllData()
-    {
-        var db = _multi.GetDatabase();
-        db.KeyDelete(_key);
-    }
-
-    public Dictionary<StreamDataKey, StreamData?> CacheGetAllData()
-    {
-        var db = _multi.GetDatabase();
-        if (!db.KeyExists(_key))
-            return new();
-
-        return db.HashGetAll(_key)
-                 .ToDictionary(entry => JsonConvert.DeserializeObject<StreamDataKey>(entry.Name),
-                     entry => entry.Value.IsNullOrEmpty
-                         ? default
-                         : JsonConvert.DeserializeObject<StreamData>(entry.Value));
-    }
+    public Dictionary<StreamDataKey, StreamData?> GetAllData()
+        => _cache.ToDictionary(x => x.Key, x => x.Value);
 
     public async Task<StreamData?> GetStreamDataByUrlAsync(string url)
     {
@@ -234,9 +207,9 @@ public class NotifChecker
 
         // if stream is found, add it to the cache for tracking only if it doesn't already exist
         // because stream will be checked and events will fire in a loop. We don't want to override old state
-        return CacheAddData(data.CreateKey(), data, false);
+        return AddLastData(data.CreateKey(), data, false);
     }
 
     public void UntrackStreamByKey(in StreamDataKey key)
-        => CacheDeleteData(key);
+        => DeleteLastData(key);
 }

src/NadekoBot/Modules/Utility/Patronage/Config/PatronConfigData.cs

@@ -7,7 +7,7 @@ namespace NadekoBot.Modules.Utility.Patronage;
 public partial class PatronConfigData : ICloneable<PatronConfigData>
 {
     [Comment("DO NOT CHANGE")]
-    public int Version { get; set; } = 1;
+    public int Version { get; set; } = 2;
     
     [Comment("Whether the patronage feature is enabled")]
     public bool IsEnabled { get; set; }

src/NadekoBot/Modules/Utility/Patronage/Config/PatronageConfig.cs

@@ -14,5 +14,23 @@ public class PatronageConfig : ConfigServiceBase<PatronConfigData>
 
     public PatronageConfig(IConfigSeria serializer, IPubSub pubSub) : base(FILE_PATH, serializer, pubSub, _changeKey)
     {
+        AddParsedProp("enabled",
+            x => x.IsEnabled,
+            bool.TryParse,
+            ConfigPrinters.ToString);
+
+        Migrate();
+    }
+
+    private void Migrate()
+    {
+        ModifyConfig(c =>
+        {
+            if (c.Version == 1)
+            {
+                c.Version = 2;
+                c.IsEnabled = false;
+            }
+        });
     }
 }

src/NadekoBot/Modules/Utility/Patronage/CurrencyRewardService.cs

@@ -50,67 +50,91 @@ public class CurrencyRewardService : INService, IDisposable
 
     private async Task OnPatronUpdate(Patron oldPatron, Patron newPatron)
     {
-        if (oldPatron.Amount != newPatron.Amount)
+        // if pledge was increased
+        if (oldPatron.Amount < newPatron.Amount)
         {
             var conf = _config.Data;
+            var newAmount = (long)(newPatron.Amount * conf.PatreonCurrencyPerCent);
 
-            var newAmount = (long)(Math.Max(newPatron.Amount, oldPatron.Amount) * conf.PatreonCurrencyPerCent);
-            UpdateOutput<RewardedUser>[] output;
+            RewardedUser old;
             await using (var ctx = _db.GetDbContext())
             {
-                output = await ctx.GetTable<RewardedUser>()
-                                  .Where(x => x.PlatformUserId == newPatron.UnqiuePlatformUserId)
-                                  .UpdateWithOutputAsync(old => new()
+                old = await ctx.GetTable<RewardedUser>()
+                         .Where(x => x.PlatformUserId == newPatron.UniquePlatformUserId)
+                         .FirstOrDefaultAsync();
+                
+                if (old is null)
+                {
+                    await OnNewPayment(newPatron);
+                    return;
+                }
+                
+                // no action as the amount is the same or lower
+                if (old.AmountRewardedThisMonth >= newAmount)
+                    return;
+                
+                var count = await ctx.GetTable<RewardedUser>()
+                                  .Where(x => x.PlatformUserId == newPatron.UniquePlatformUserId)
+                                  .UpdateAsync(_ => new()
                                   {
-                                      PlatformUserId = newPatron.UnqiuePlatformUserId,
+                                      PlatformUserId = newPatron.UniquePlatformUserId,
                                       UserId = newPatron.UserId,
                                       // amount before bonuses
                                       AmountRewardedThisMonth = newAmount,
                                       LastReward = newPatron.PaidAt
                                   });
+
+                // shouldn't ever happen
+                if (count == 0)
+                    return;
             }
 
-            // if the user wasn't previously in the db for some reason,
-            // we will treat him as a new patron
-            if (output.Length == 0)
-            {
-                await OnNewPayment(newPatron);
-                return;
-            }
+            var oldAmount = old.AmountRewardedThisMonth;
 
-            var oldAmount = output[0].Deleted.AmountRewardedThisMonth;
-
-            var diff = newAmount - oldAmount;
+            var realNewAmount = GetRealCurrencyReward(
+                (int)(newAmount / conf.PatreonCurrencyPerCent),
+                newAmount,
+                out var percentBonus);
+            
+            var realOldAmount = GetRealCurrencyReward(
+                (int)(oldAmount / conf.PatreonCurrencyPerCent),
+                oldAmount,
+                out _);
+            
+            var diff = realNewAmount - realOldAmount;
             if (diff <= 0)
                 return; // no action if new is lower
 
             // if the user pledges 5$ or more, they will get X % more flowers where X is amount in dollars,
             // up to 100%
-
-            var realAmount = GetRealCurrencyReward(newPatron.Amount, diff, out var percentBonus);
-            await _cs.AddAsync(newPatron.UserId, realAmount, new TxData("patron","update"));
+            
+            await _cs.AddAsync(newPatron.UserId, diff, new TxData("patron","update"));
             
             _ = SendMessageToUser(newPatron.UserId,
-                $"You've received an additional **{realAmount}**{_config.Data.Currency.Sign} as a currency reward (+{percentBonus}%)!");
+                $"You've received an additional **{diff}**{_config.Data.Currency.Sign} as a currency reward (+{percentBonus}%)!");
         }
     }
 
-    private long GetRealCurrencyReward(int fullPledge, long currentAmount, out int percentBonus)
+    private long GetRealCurrencyReward(int pledgeCents, long modifiedAmount, out int percentBonus)
     {
         // needs at least 5$ to be eligible for a bonus
-        if (fullPledge < 500)
+        if (pledgeCents < 500)
         {
             percentBonus = 0;
-            return currentAmount;
+            return modifiedAmount;
         }
 
-        var dollarValue = fullPledge / 100;
+        var dollarValue = pledgeCents / 100;
         percentBonus = dollarValue switch
         {
-            > 100 => 100,
-            _ => dollarValue
+            >= 100 => 100,
+            >= 50 => 50,
+            >= 20 => 20,
+            >= 10 => 10,
+            >= 5 => 5,
+            _ => 0
         };
-        return (long)(currentAmount * (1 + (percentBonus / 100.0f)));
+        return (long)(modifiedAmount * (1 + (percentBonus / 100.0f)));
     }
 
     // on a new payment, always give the full amount.
@@ -121,7 +145,7 @@ public class CurrencyRewardService : INService, IDisposable
         await ctx.GetTable<RewardedUser>()
                  .InsertOrUpdateAsync(() => new()
                      {
-                         PlatformUserId = patron.UnqiuePlatformUserId,
+                         PlatformUserId = patron.UniquePlatformUserId,
                          UserId = patron.UserId,
                          AmountRewardedThisMonth = amount,
                          LastReward = patron.PaidAt,
@@ -134,7 +158,7 @@ public class CurrencyRewardService : INService, IDisposable
                      },
                      () => new()
                      {
-                         PlatformUserId = patron.UnqiuePlatformUserId
+                         PlatformUserId = patron.UniquePlatformUserId
                      });
         
         var realAmount = GetRealCurrencyReward(patron.Amount, amount, out var percentBonus);
@@ -167,24 +191,9 @@ public class CurrencyRewardService : INService, IDisposable
     {
         await using var ctx = _db.GetDbContext();
         _ = await ctx.GetTable<RewardedUser>()
-                     .UpdateWithOutputAsync(old => new()
+                     .UpdateAsync(old => new()
                      {
                          AmountRewardedThisMonth = old.AmountRewardedThisMonth * 2
                      });
-
-        // var toTake = old.Length == 0
-        //     ? patron.Amount
-        //     : old[0].Inserted.AmountRewardedThisMonth;
-
-        // if (toTake > 0)
-        // {
-        //     Log.Warning("Wiping the wallet and bank of the user {UserId} due to a refund/fraud...",
-        //         patron.UserId);
-        //     await _cs.RemoveAsync(patron.UserId, patron.Amount, new("patreon", "refund"));
-        //     await _bs.BurnAllAsync(patron.UserId);
-        //     Log.Warning("Burned {Amount} currency from the bank of the user {UserId} due to a refund/fraud.",
-        //         patron.Amount,
-        //         patron.UserId);
-        // }
     }
 }

src/NadekoBot/Modules/Utility/Patronage/Patron.cs

@@ -5,10 +5,10 @@ public readonly struct Patron
     /// <summary>
     /// Unique id assigned to this patron by the payment platform
     /// </summary>
-    public string UnqiuePlatformUserId { get; init; }
+    public string UniquePlatformUserId { get; init; }
 
     /// <summary>
-    /// Discord UserId to which this <see cref="UnqiuePlatformUserId"/> is connected to
+    /// Discord UserId to which this <see cref="UniquePlatformUserId"/> is connected to
     /// </summary>
     public ulong UserId { get; init; }
 

src/NadekoBot/Modules/Utility/Patronage/PatronageService.cs

@@ -3,6 +3,7 @@ using LinqToDB.EntityFrameworkCore;
 using NadekoBot.Common.ModuleBehaviors;
 using NadekoBot.Db.Models;
 using OneOf;
+using OneOf.Types;
 using StackExchange.Redis;
 using CommandInfo = Discord.Commands.CommandInfo;
 
@@ -30,9 +31,10 @@ public sealed class PatronageService
     private readonly DiscordSocketClient _client;
     private readonly ISubscriptionHandler _subsHandler;
     private readonly IEmbedBuilderService _eb;
-    private readonly ConnectionMultiplexer _redis;
-    private readonly IBotCredentials _creds;
-    private readonly TypedKey<bool> _quotaKey;
+    private static readonly TypedKey<long> _quotaKey 
+        = new($"quota:last_hourly_reset");
+
+    private readonly IBotCache _cache;
 
     public PatronageService(
         PatronageConfig pConf,
@@ -40,18 +42,14 @@ public sealed class PatronageService
         DiscordSocketClient client,
         ISubscriptionHandler subsHandler,
         IEmbedBuilderService eb,
-        ConnectionMultiplexer redis,
-        IBotCredentials creds)
+        IBotCache cache)
     {
         _pConf = pConf;
         _db = db;
         _client = client;
         _subsHandler = subsHandler;
         _eb = eb;
-        _redis = redis;
-        _creds = creds;
-
-        _quotaKey = new TypedKey<bool>($"{_creds.RedisKey()}:quota:last_hourly_reset");
+        _cache = cache;
     }
 
     public Task OnReadyAsync()
@@ -100,11 +98,10 @@ public sealed class PatronageService
                 var now = DateTime.UtcNow;
                 var lastRun = DateTime.MinValue;
 
-                var rdb = _redis.GetDatabase();
-                var lastVal = await rdb.StringGetAsync(_quotaKey.Key);
-                if (lastVal != default)
+                var result = await _cache.GetAsync(_quotaKey);
+                if (result.TryGetValue(out var lastVal) && lastVal != default)
                 {
-                    lastRun = DateTime.FromBinary((long)lastVal);
+                    lastRun = DateTime.FromBinary(lastVal);
                 }
 
                 var nowDate = now.ToDateOnly();
@@ -129,8 +126,6 @@ public sealed class PatronageService
                                  HourlyCount = 0,
                                  DailyCount = 0,
                              });
-
-                    await rdb.StringSetAsync(_quotaKey.Key, true);
                 }
                 else if (now.Hour != lastRun.Hour) // if it's not, just reset hourly quotas
                 {
@@ -142,7 +137,7 @@ public sealed class PatronageService
                 }
 
                 // assumes that the code above runs in less than an hour
-                await rdb.StringSetAsync(_quotaKey.Key, now.ToBinary());
+                await _cache.AddAsync(_quotaKey, now.ToBinary());
                 await tran.CommitAsync();
             }
             catch (Exception ex)
@@ -173,7 +168,7 @@ public sealed class PatronageService
         
             var lastChargeUtc = subscriber.LastCharge.Value.ToUniversalTime();
             var dateInOneMonth = lastChargeUtc.Date.AddMonths(1);
-            await using var tran = await ctx.Database.BeginTransactionAsync();
+            // await using var tran = await ctx.Database.BeginTransactionAsync();
             try
             {
                 var dbPatron = await ctx.GetTable<PatronUser>()
@@ -193,7 +188,7 @@ public sealed class PatronageService
                                             ValidThru = dateInOneMonth,
                                         });
         
-                    await tran.CommitAsync();
+                    // await tran.CommitAsync();
         
                     var newPatron = PatronUserToPatron(dbPatron);
                     _ = SendWelcomeMessage(newPatron);
@@ -222,35 +217,38 @@ public sealed class PatronageService
                         // this should never happen
                         if (count == 0)
                         {
-                            await tran.RollbackAsync();
+                            // await tran.RollbackAsync();
                             continue;
                         }
         
-                        await tran.CommitAsync();
+                        // await tran.CommitAsync();
         
                         await OnNewPatronPayment(PatronUserToPatron(dbPatron));
                     }
                     else if (dbPatron.AmountCents != subscriber.Cents // if user changed the amount 
                              || dbPatron.UserId != subscriber.UserId) // if user updated user id)
                     {
+                        var cents = subscriber.Cents;
                         // the user updated the pledge or changed the connected discord account
-                        var newData = await ctx.GetTable<PatronUser>()
-                                               .Where(x => x.UniquePlatformUserId == subscriber.UniquePlatformUserId
-                                                           && x.LastCharge < lastChargeUtc)
-                                               .UpdateWithOutputAsync(old => new()
-                                               {
-                                                   UserId = subscriber.UserId,
-                                                   AmountCents = subscriber.Cents,
-                                                   LastCharge = lastChargeUtc,
-                                                   ValidThru = old.ValidThru,
-                                               });
-                        await tran.CommitAsync();
-        
-                        // this should never happen
-                        if (newData.Length == 0)
-                            continue;
-        
-                        await OnPatronUpdated(PatronUserToPatron(dbPatron), PatronUserToPatron(newData[0].Inserted));
+                        await ctx.GetTable<PatronUser>()
+                                 .Where(x => x.UniquePlatformUserId == subscriber.UniquePlatformUserId)
+                                 .UpdateAsync(old => new()
+                                 {
+                                     UserId = subscriber.UserId,
+                                     AmountCents = cents,
+                                     LastCharge = lastChargeUtc,
+                                     ValidThru = old.ValidThru,
+                                 });
+
+                        var newPatron = dbPatron.Clone();
+                        newPatron.AmountCents = cents;
+                        newPatron.UserId = subscriber.UserId;
+                        
+                        // idk what's going on but UpdateWithOutputAsync doesn't work properly here
+                        // nor does firstordefault after update. I'm not seeing something obvious
+                        await OnPatronUpdated(
+                            PatronUserToPatron(dbPatron),
+                            PatronUserToPatron(newPatron));
                     }
                 }
             }
@@ -262,22 +260,26 @@ public sealed class PatronageService
             }
         }
         
+        var expiredDate = DateTime.MinValue;
         foreach (var patron in subscribers.Where(x => x.ChargeStatus == SubscriptionChargeStatus.Refunded))
         {
-            var expiredDate = DateTime.MinValue;
             // if the subscription is refunded, Disable user's valid thru 
-            var output = await ctx.GetTable<PatronUser>()
+            var changedCount = await ctx.GetTable<PatronUser>()
                                   .Where(x => x.UniquePlatformUserId == patron.UniquePlatformUserId
                                               && x.ValidThru != expiredDate)
-                                  .UpdateWithOutputAsync(old => new()
+                                  .UpdateAsync(old => new()
                                   {
                                       ValidThru = expiredDate
                                   });
         
-            if (output.Length == 0)
+            if (changedCount == 0)
                 continue;
-        
-            await OnPatronRefunded(PatronUserToPatron(output[0].Inserted));
+
+            var updated = await ctx.GetTable<PatronUser>()
+                                  .Where(x => x.UniquePlatformUserId == patron.UniquePlatformUserId)
+                                  .FirstAsync();
+            
+            await OnPatronRefunded(PatronUserToPatron(updated));
         }
     }
 
@@ -493,8 +495,8 @@ public sealed class PatronageService
         if (!confData.IsEnabled)
             return default;
 
-        if (_creds.IsOwner(userId))
-            return default;
+        // if (_creds.IsOwner(userId))
+        //     return default;
 
         // get user tier
         var patron = await GetPatronAsync(userId);
@@ -551,7 +553,9 @@ public sealed class PatronageService
             data.TryGetValue(QuotaPer.PerMonth, out var monthly) ? monthly : null
         );
 
-        return quotaCheckResult.Match(_ => default, x => x);
+        return quotaCheckResult.Match<OneOf<Success, InsufficientTier, QuotaLimit>>(
+            _ => new Success(),
+            x => x);
     }
 
     private bool TryGetTierDataOrLower<T>(
@@ -641,7 +645,6 @@ public sealed class PatronageService
         };
     }
 
-    // should i allow users to pay extra for more quota?
     private IReadOnlyDictionary<string, FeatureQuotaStats> GetFeatureQuotaStats(
         PatronTier patronTier,
         IReadOnlyDictionary<string, PatronQuota>? allQuotasDict,
@@ -691,7 +694,7 @@ public sealed class PatronageService
             return new()
             {
                 Name = key.PrettyName,
-                Quota = default,
+                Quota = defaultValue,
                 IsPatronLimit = false
             };
         
@@ -732,7 +735,7 @@ public sealed class PatronageService
     private Patron PatronUserToPatron(PatronUser user)
         => new Patron()
         {
-            UnqiuePlatformUserId = user.UniquePlatformUserId,
+            UniquePlatformUserId = user.UniquePlatformUserId,
             UserId = user.UserId,
             Amount = user.AmountCents,
             Tier = CalculateTier(user),
@@ -747,6 +750,9 @@ public sealed class PatronageService
 
         return user.AmountCents switch
         {
+            >= 10_000 => PatronTier.C,
+            >= 5000 => PatronTier.L,
+            >= 2000 => PatronTier.XX,
             >= 1000 => PatronTier.X,
             >= 500 => PatronTier.V,
             >= 100 => PatronTier.I,
@@ -782,7 +788,7 @@ public sealed class PatronageService
 *- Any user in any of your servers can use Patron-only commands, but they will spend **your quota**, which is why it's recommended to use Nadeko's command cooldown system (.h .cmdcd) or permission system to limit the command usage for your server members.*
 *- Permission guide can be found here if you're not familiar with it: <https://nadekobot.readthedocs.io/en/latest/permissions-system/>*",
                             isInline: false)
-                        .WithFooter($"platform id: {patron.UnqiuePlatformUserId}");
+                        .WithFooter($"platform id: {patron.UniquePlatformUserId}");
 
             await user.EmbedAsync(eb);
         }

src/NadekoBot/Modules/Utility/StreamRole/StreamRoleService.cs

@@ -1,17 +1,20 @@
 #nullable disable
+using NadekoBot.Common.ModuleBehaviors;
 using NadekoBot.Db;
 using NadekoBot.Modules.Utility.Common;
 using NadekoBot.Modules.Utility.Common.Exceptions;
 using NadekoBot.Services.Database.Models;
+using System.Diagnostics;
 using System.Net;
 
 namespace NadekoBot.Modules.Utility.Services;
 
-public class StreamRoleService : INService
+public class StreamRoleService : IReadyExecutor, INService
 {
     private readonly DbService _db;
     private readonly DiscordSocketClient _client;
     private readonly ConcurrentDictionary<ulong, StreamRoleSettings> _guildSettings;
+    private readonly QueueRunner _queueRunner;
 
     public StreamRoleService(DiscordSocketClient client, DbService db, Bot bot)
     {
@@ -22,33 +25,35 @@ public class StreamRoleService : INService
                             .Where(x => x.Value is { Enabled: true })
                             .ToConcurrent();
 
-        _client.GuildMemberUpdated += Client_GuildMemberUpdated;
+        _client.PresenceUpdated += OnPresenceUpdate;
 
-        _ = Task.Run(async () =>
-        {
-            try
-            {
-                await client.Guilds.Select(g => RescanUsers(g)).WhenAll();
-            }
-            catch
-            {
-                // ignored
-            }
-        });
+        _queueRunner = new QueueRunner();
     }
 
-    private Task Client_GuildMemberUpdated(Cacheable<SocketGuildUser, ulong> cacheable, SocketGuildUser after)
+    private Task OnPresenceUpdate(SocketUser user, SocketPresence oldPresence, SocketPresence newPresence)
     {
+        
         _ = Task.Run(async () =>
         {
-            //if user wasn't streaming or didn't have a game status at all
-            if (_guildSettings.TryGetValue(after.Guild.Id, out var setting))
-                await RescanUser(after, setting);
+            if (oldPresence.Activities.Count != newPresence.Activities.Count)
+            {
+                var guildUsers = _client.Guilds
+                                        .Select(x => x.GetUser(user.Id));
+
+                foreach (var guildUser in guildUsers)
+                {
+                    if (_guildSettings.TryGetValue(guildUser.Guild.Id, out var s))
+                        await RescanUser(guildUser, s);
+                }
+            }
         });
 
         return Task.CompletedTask;
     }
 
+    public Task OnReadyAsync()
+        => Task.WhenAll(_client.Guilds.Select(RescanUsers).WhenAll(), _queueRunner.RunAsync());
+
     /// <summary>
     ///     Adds or removes a user from a blacklist or a whitelist in the specified guild.
     /// </summary>
@@ -135,7 +140,7 @@ public class StreamRoleService : INService
 
             streamRoleSettings.Keyword = keyword;
             UpdateCache(guild.Id, streamRoleSettings);
-            uow.SaveChanges();
+            await uow.SaveChangesAsync();
         }
 
         await RescanUsers(guild);
@@ -191,8 +196,7 @@ public class StreamRoleService : INService
 
         foreach (var usr in await fromRole.GetMembersAsync())
         {
-            if (usr is { } x)
-                await RescanUser(x, setting, addRole);
+            await RescanUser(usr, setting, addRole);
         }
     }
 
@@ -216,7 +220,10 @@ public class StreamRoleService : INService
             await RescanUsers(guild);
     }
 
-    private async Task RescanUser(IGuildUser user, StreamRoleSettings setting, IRole addRole = null)
+    private async ValueTask RescanUser(IGuildUser user, StreamRoleSettings setting, IRole addRole = null)
+        => await _queueRunner.Enqueue(() => RescanUserInternal(user, setting, addRole));
+
+    private async Task RescanUserInternal(IGuildUser user, StreamRoleSettings setting, IRole addRole = null)
     {
         if (user.IsBot)
             return;
@@ -232,58 +239,77 @@ public class StreamRoleService : INService
             && setting.Blacklist.All(x => x.UserId != user.Id)
             && user.RoleIds.Contains(setting.FromRoleId))
         {
-            try
+            await _queueRunner.Enqueue(async () =>
             {
-                addRole ??= user.Guild.GetRole(setting.AddRoleId);
-                if (addRole is null)
+                try
+                {
+                    addRole ??= user.Guild.GetRole(setting.AddRoleId);
+                    if (addRole is null)
+                    {
+                        await StopStreamRole(user.Guild);
+                        Log.Warning("Stream role in server {RoleId} no longer exists. Stopping", setting.AddRoleId);
+                        return;
+                    }
+
+                    //check if he doesn't have addrole already, to avoid errors
+                    if (!user.RoleIds.Contains(addRole.Id))
+                    {
+                        await user.AddRoleAsync(addRole);
+                        Log.Information("Added stream role to user {User} in {Server} server",
+                            user.ToString(),
+                            user.Guild.ToString());
+                    }
+                }
+                catch (HttpException ex) when (ex.HttpCode == HttpStatusCode.Forbidden)
                 {
                     await StopStreamRole(user.Guild);
-                    Log.Warning("Stream role in server {RoleId} no longer exists. Stopping", setting.AddRoleId);
-                    return;
+                    Log.Warning(ex, "Error adding stream role(s). Forcibly disabling stream role feature");
+                    throw new StreamRolePermissionException();
                 }
-
-                //check if he doesn't have addrole already, to avoid errors
-                if (!user.RoleIds.Contains(addRole.Id))
+                catch (Exception ex)
                 {
-                    await user.AddRoleAsync(addRole);
-                    Log.Information("Added stream role to user {User} in {Server} server",
-                        user.ToString(),
-                        user.Guild.ToString());
+                    Log.Warning(ex, "Failed adding stream role");
                 }
-            }
-            catch (HttpException ex) when (ex.HttpCode == HttpStatusCode.Forbidden)
-            {
-                await StopStreamRole(user.Guild);
-                Log.Warning(ex, "Error adding stream role(s). Forcibly disabling stream role feature");
-                throw new StreamRolePermissionException();
-            }
-            catch (Exception ex)
-            {
-                Log.Warning(ex, "Failed adding stream role");
-            }
+            });
         }
         else
         {
             //check if user is in the addrole
             if (user.RoleIds.Contains(setting.AddRoleId))
             {
-                try
+                await _queueRunner.Enqueue(async () =>
                 {
-                    addRole ??= user.Guild.GetRole(setting.AddRoleId);
-                    if (addRole is null)
-                        throw new StreamRoleNotFoundException();
+                    try
+                    {
+                        addRole ??= user.Guild.GetRole(setting.AddRoleId);
+                        if (addRole is null)
+                        {
+                            await StopStreamRole(user.Guild);
+                            Log.Warning(
+                                "Addrole doesn't exist in {GuildId} server. Forcibly disabling stream role feature",
+                                user.Guild.Id);
+                            return;
+                        }
 
-                    await user.RemoveRoleAsync(addRole);
-                    Log.Information("Removed stream role from the user {User} in {Server} server",
-                        user.ToString(),
-                        user.Guild.ToString());
-                }
-                catch (HttpException ex) when (ex.HttpCode == HttpStatusCode.Forbidden)
-                {
-                    await StopStreamRole(user.Guild);
-                    Log.Warning(ex, "Error removing stream role(s). Forcibly disabling stream role feature");
-                    throw new StreamRolePermissionException();
-                }
+                        // need to check again in case queuer is taking too long to execute
+                        if (user.RoleIds.Contains(setting.AddRoleId))
+                        {
+                            await user.RemoveRoleAsync(addRole);
+                        }
+
+                        Log.Information("Removed stream role from the user {User} in {Server} server",
+                            user.ToString(),
+                            user.Guild.ToString());
+                    }
+                    catch (HttpException ex)
+                    {
+                        if (ex.HttpCode == HttpStatusCode.Forbidden)
+                        {
+                            await StopStreamRole(user.Guild);
+                            Log.Warning(ex, "Error removing stream role(s). Forcibly disabling stream role feature");
+                        }
+                    }
+                });
             }
         }
     }

src/NadekoBot/Modules/Utility/UnitConversion/ConverterService.cs

@@ -1,23 +1,24 @@
 #nullable disable
 using NadekoBot.Common.ModuleBehaviors;
 using NadekoBot.Modules.Utility.Common;
-using Newtonsoft.Json;
+using System.Text.Json;
+using System.Text.Json.Serialization;
 
 namespace NadekoBot.Modules.Utility.Services;
 
 public class ConverterService : INService, IReadyExecutor
 {
-    public ConvertUnit[] Units
-        => _cache.Redis.GetDatabase().StringGet("converter_units").ToString().MapJson<ConvertUnit[]>();
+    private static readonly TypedKey<List<ConvertUnit>> _convertKey =
+        new("convert:units");
 
     private readonly TimeSpan _updateInterval = new(12, 0, 0);
     private readonly DiscordSocketClient _client;
-    private readonly IDataCache _cache;
+    private readonly IBotCache _cache;
     private readonly IHttpClientFactory _httpFactory;
 
     public ConverterService(
         DiscordSocketClient client,
-        IDataCache cache,
+        IBotCache cache,
         IHttpClientFactory factory)
     {
         _client = client;
@@ -48,7 +49,7 @@ public class ConverterService : INService, IReadyExecutor
     {
         using var http = _httpFactory.CreateClient();
         var res = await http.GetStringAsync("https://convertapi.nadeko.bot/latest");
-        return JsonConvert.DeserializeObject<Rates>(res);
+        return JsonSerializer.Deserialize<Rates>(res);
     }
 
     private async Task UpdateCurrency()
@@ -61,29 +62,38 @@ public class ConverterService : INService, IReadyExecutor
             Modifier = decimal.One,
             UnitType = unitTypeString
         };
-        var range = currencyRates.ConversionRates.Select(u => new ConvertUnit
+        var units = currencyRates.ConversionRates.Select(u => new ConvertUnit
                                  {
                                      Triggers = new[] { u.Key },
                                      Modifier = u.Value,
                                      UnitType = unitTypeString
                                  })
-                                 .ToArray();
+                                 .ToList();
 
-        var fileData = JsonConvert.DeserializeObject<ConvertUnit[]>(File.ReadAllText("data/units.json"))
-                                  ?.Where(x => x.UnitType != "currency");
-        if (fileData is null)
-            return;
-
-        var data = JsonConvert.SerializeObject(range.Append(baseType).Concat(fileData).ToList());
-        _cache.Redis.GetDatabase().StringSet("converter_units", data);
+        var stream =  File.OpenRead("data/units.json");
+        var defaultUnits = await JsonSerializer.DeserializeAsync<ConvertUnit[]>(stream);
+        if(defaultUnits is not null)
+            units.AddRange(defaultUnits);
+        
+        units.Add(baseType);
+        
+        await _cache.AddAsync(_convertKey, units);
     }
+
+    public async Task<IReadOnlyList<ConvertUnit>> GetUnitsAsync()
+        => (await _cache.GetAsync(_convertKey)).TryGetValue(out var list)
+            ? list
+            : Array.Empty<ConvertUnit>();
 }
 
 public class Rates
 {
+    [JsonPropertyName("base")]
     public string Base { get; set; }
+    
+    [JsonPropertyName("date")]
     public DateTime Date { get; set; }
 
-    [JsonProperty("rates")]
+    [JsonPropertyName("rates")]
     public Dictionary<string, decimal> ConversionRates { get; set; }
 }

src/NadekoBot/Modules/Utility/UnitConversion/UnitConversionCommands.cs

@@ -11,7 +11,7 @@ public partial class Utility
         [Cmd]
         public async partial Task ConvertList()
         {
-            var units = _service.Units;
+            var units = await _service.GetUnitsAsync();
 
             var embed = _eb.Create().WithTitle(GetText(strs.convertlist)).WithOkColor();
 
@@ -29,9 +29,10 @@ public partial class Utility
         [Priority(0)]
         public async partial Task Convert(string origin, string target, decimal value)
         {
-            var originUnit = _service.Units.FirstOrDefault(x
+            var units = await _service.GetUnitsAsync();
+            var originUnit = units.FirstOrDefault(x
                 => x.Triggers.Select(y => y.ToUpperInvariant()).Contains(origin.ToUpperInvariant()));
-            var targetUnit = _service.Units.FirstOrDefault(x
+            var targetUnit = units.FirstOrDefault(x
                 => x.Triggers.Select(y => y.ToUpperInvariant()).Contains(target.ToUpperInvariant()));
             if (originUnit is null || targetUnit is null)
             {

src/NadekoBot/Modules/Utility/Utility.cs

@@ -1,4 +1,7 @@
 #nullable disable
+using Microsoft.CodeAnalysis.CSharp.Scripting;
+using Microsoft.CodeAnalysis.Scripting;
+using NadekoBot.Modules.Utility.Services;
 using Newtonsoft.Json;
 using System.Diagnostics;
 using System.Text;
@@ -32,6 +35,7 @@ public partial class Utility : NadekoModule
     private readonly IBotCredentials _creds;
     private readonly DownloadTracker _tracker;
     private readonly IHttpClientFactory _httpFactory;
+    private readonly VerboseErrorsService _veService;
 
     public Utility(
         DiscordSocketClient client,
@@ -39,7 +43,8 @@ public partial class Utility : NadekoModule
         IStatsService stats,
         IBotCredentials creds,
         DownloadTracker tracker,
-        IHttpClientFactory httpFactory)
+        IHttpClientFactory httpFactory,
+        VerboseErrorsService veService)
     {
         _client = client;
         _coord = coord;
@@ -47,6 +52,7 @@ public partial class Utility : NadekoModule
         _creds = creds;
         _tracker = tracker;
         _httpFactory = httpFactory;
+        _veService = veService;
     }
     
     [Cmd]
@@ -482,44 +488,17 @@ public partial class Utility : NadekoModule
             sem.Release();
         }
     }
+    
+    [Cmd]
+    [RequireContext(ContextType.Guild)]
+    [UserPerm(GuildPerm.ManageMessages)]
+    public async partial Task VerboseError(bool? newstate = null)
+    {
+        var state = _veService.ToggleVerboseErrors(ctx.Guild.Id, newstate);
 
-
-    // [NadekoCommand, Usage, Description, Aliases]
-    // [RequireContext(ContextType.Guild)]
-    // public async Task CreateMyInvite(CreateInviteType type = CreateInviteType.Any)
-    // {
-    //     if (type == CreateInviteType.Any)
-    //     {
-    //         if (_inviteService.TryGetInvite(type, out var code))
-    //         {
-    //             await ReplyErrorLocalizedAsync(strs.your_invite($"https://discord.gg/{code}"));
-    //             return;
-    //         }
-    //     }
-    //     
-    //     var invite = await ((ITextChannel) ctx.Channel).CreateInviteAsync(isUnique: true);
-    // }
-    //
-    // [NadekoCommand, Usage, Description, Aliases]
-    // [RequireContext(ContextType.Guild)]
-    // public async partial Task InviteLb(int page = 1)
-    // {
-    //     if (--page < 0)
-    //         return;
-    //
-    //     var inviteUsers = await _inviteService.GetInviteUsersAsync(ctx.Guild.Id);
-    //     
-    //     var embed = _eb.Create()
-    //         .WithOkColor();
-    //
-    //     await ctx.SendPaginatedConfirmAsync(page, (curPage) =>
-    //     {
-    //         var items = inviteUsers.Skip(curPage * 9).Take(9);
-    //         var i = 0;
-    //         foreach (var item in items)
-    //             embed.AddField($"#{curPage * 9 + ++i} {item.UserName} [{item.User.Id}]", item.InvitedUsers);
-    //
-    //         return embed;
-    //     }, inviteUsers.Count, 9);
-    // }
+        if (state)
+            await ReplyConfirmLocalizedAsync(strs.verbose_errors_enabled);
+        else
+            await ReplyConfirmLocalizedAsync(strs.verbose_errors_disabled);
+    }
 }

src/NadekoBot/Modules/Utility/VerboseErrors/EvalCommands.cs

@@ -0,0 +1,76 @@
+#nullable disable
+using Microsoft.CodeAnalysis.CSharp.Scripting;
+using Microsoft.CodeAnalysis.Scripting;
+
+namespace NadekoBot.Modules.Utility;
+
+public partial class Utility
+{
+    [Group]
+    public partial class EvalCommands : NadekoModule
+    {
+        private readonly IServiceProvider _services;
+
+        public EvalCommands(IServiceProvider services)
+        {
+            _services = services;
+        }
+        
+        [Cmd]
+        [NoPublicBot]
+        [OwnerOnly]
+        public async partial Task Eval([Leftover] string scriptText)
+        {
+            _ = ctx.Channel.TriggerTypingAsync();
+
+            if (scriptText.StartsWith("```cs"))
+                scriptText = scriptText[5..];
+            else if (scriptText.StartsWith("```"))
+                scriptText = scriptText[3..];
+
+            if (scriptText.EndsWith("```"))
+                scriptText = scriptText[..^3];
+            
+            var script = CSharpScript.Create(scriptText,
+                ScriptOptions.Default
+                             .WithReferences(this.GetType().Assembly)
+                             .WithImports(
+                                 "System",
+                                 "NadekoBot",
+                                 "NadekoBot.Extensions",
+                                 "Microsoft.Extensions.DependencyInjection",
+                                 "NadekoBot.Common",
+                                 "System.Text",
+                                 "System.Text.Json"),
+                globalsType: typeof(EvalGlobals));
+            
+            try
+            {
+                var result = await script.RunAsync(new EvalGlobals()
+                {
+                    ctx = this.ctx,
+                    guild = this.ctx.Guild,
+                    channel = this.ctx.Channel,
+                    user = this.ctx.User,
+                    self = this,
+                    services = _services
+                });
+
+                var output = result.ReturnValue?.ToString();
+                if (!string.IsNullOrWhiteSpace(output))
+                {
+                    var eb = _eb.Create(ctx)
+                                .WithOkColor()
+                                .AddField("Code", scriptText)
+                                .AddField("Output", output.TrimTo(512)!);
+
+                    _ = ctx.Channel.EmbedAsync(eb);
+                }
+            }
+            catch (Exception ex)
+            {
+                await SendErrorAsync(ex.Message);
+            }
+        }
+    }
+}

src/NadekoBot/Modules/Utility/VerboseErrors/EvalGlobals.cs

@@ -0,0 +1,13 @@
+// ReSharper disable InconsistentNaming
+#nullable disable
+namespace NadekoBot.Modules.Utility;
+
+public class EvalGlobals
+{
+    public ICommandContext ctx;
+    public Utility.EvalCommands self;
+    public IUser user;
+    public IMessageChannel channel;
+    public IGuild guild;
+    public IServiceProvider services;
+}

src/NadekoBot/Modules/Utility/VerboseErrors/VerboseErrorCommands.cs

@@ -1,24 +0,0 @@
-#nullable disable
-using NadekoBot.Modules.Utility.Services;
-
-namespace NadekoBot.Modules.Utility;
-
-public partial class Utility
-{
-    [Group]
-    public partial class VerboseErrorCommands : NadekoModule<VerboseErrorsService>
-    {
-        [Cmd]
-        [RequireContext(ContextType.Guild)]
-        [UserPerm(GuildPerm.ManageMessages)]
-        public async partial Task VerboseError(bool? newstate = null)
-        {
-            var state = _service.ToggleVerboseErrors(ctx.Guild.Id, newstate);
-
-            if (state)
-                await ReplyConfirmLocalizedAsync(strs.verbose_errors_enabled);
-            else
-                await ReplyConfirmLocalizedAsync(strs.verbose_errors_disabled);
-        }
-    }
-}

src/NadekoBot/Modules/Xp/XpService.cs

@@ -23,10 +23,8 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
     public const int XP_REQUIRED_LVL_1 = 36;
 
     private readonly DbService _db;
-    private readonly CommandHandler _cmd;
     private readonly IImageCache _images;
     private readonly IBotStrings _strings;
-    private readonly IDataCache _cache;
     private readonly FontProvider _fonts;
     private readonly IBotCredentials _creds;
     private readonly ICurrencyService _cs;
@@ -39,20 +37,21 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
     private readonly ConcurrentDictionary<ulong, ConcurrentHashSet<ulong>> _excludedChannels;
     private readonly ConcurrentHashSet<ulong> _excludedServers;
 
-    private readonly ConcurrentQueue<UserCacheItem> _addMessageXp = new();
+    private readonly System.Collections.Concurrent.ConcurrentQueue<UserCacheItem> _addMessageXp = new();
     private XpTemplate template;
     private readonly DiscordSocketClient _client;
 
     private readonly TypedKey<bool> _xpTemplateReloadKey;
     private readonly IPatronageService _ps;
+    private readonly IBotCache _c;
 
     public XpService(
         DiscordSocketClient client,
-        CommandHandler cmd,
         Bot bot,
         DbService db,
         IBotStrings strings,
-        IDataCache cache,
+        IImageCache images,
+        IBotCache c,
         FontProvider fonts,
         IBotCredentials creds,
         ICurrencyService cs,
@@ -63,10 +62,8 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
         IPatronageService ps)
     {
         _db = db;
-        _cmd = cmd;
-        _images = cache.LocalImages;
+        _images = images;
         _strings = strings;
-        _cache = cache;
         _fonts = fonts;
         _creds = creds;
         _cs = cs;
@@ -79,6 +76,7 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
         _client = client;
         _xpTemplateReloadKey = new("xp.template.reload");
         _ps = ps;
+        _c = c;
 
         InternalReloadXpTemplate();
 
@@ -453,10 +451,10 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
 
     private Task Client_OnGuildAvailable(SocketGuild guild)
     {
-        Task.Run(() =>
+        Task.Run(async () =>
         {
             foreach (var channel in guild.VoiceChannels)
-                ScanChannelForVoiceXp(channel);
+                await ScanChannelForVoiceXp(channel);
         });
 
         return Task.CompletedTask;
@@ -467,33 +465,33 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
         if (socketUser is not SocketGuildUser user || user.IsBot)
             return Task.CompletedTask;
 
-        _ = Task.Run(() =>
+        _ = Task.Run(async () =>
         {
             if (before.VoiceChannel is not null)
-                ScanChannelForVoiceXp(before.VoiceChannel);
+                await ScanChannelForVoiceXp(before.VoiceChannel);
 
             if (after.VoiceChannel is not null && after.VoiceChannel != before.VoiceChannel)
-                ScanChannelForVoiceXp(after.VoiceChannel);
+                await ScanChannelForVoiceXp(after.VoiceChannel);
             else if (after.VoiceChannel is null)
                 // In this case, the user left the channel and the previous for loops didn't catch
                 // it because it wasn't in any new channel. So we need to get rid of it.
-                UserLeftVoiceChannel(user, before.VoiceChannel);
+                await UserLeftVoiceChannel(user, before.VoiceChannel);
         });
 
         return Task.CompletedTask;
     }
 
-    private void ScanChannelForVoiceXp(SocketVoiceChannel channel)
+    private async Task ScanChannelForVoiceXp(SocketVoiceChannel channel)
     {
         if (ShouldTrackVoiceChannel(channel))
         {
             foreach (var user in channel.Users)
-                ScanUserForVoiceXp(user, channel);
+                await ScanUserForVoiceXp(user, channel);
         }
         else
         {
             foreach (var user in channel.Users)
-                UserLeftVoiceChannel(user, channel);
+                await UserLeftVoiceChannel(user, channel);
         }
     }
 
@@ -502,12 +500,12 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
     /// </summary>
     /// <param name="user"></param>
     /// <param name="channel"></param>
-    private void ScanUserForVoiceXp(SocketGuildUser user, SocketVoiceChannel channel)
+    private async Task ScanUserForVoiceXp(SocketGuildUser user, SocketVoiceChannel channel)
     {
         if (UserParticipatingInVoiceChannel(user) && ShouldTrackXp(user, channel.Id))
-            UserJoinedVoiceChannel(user);
+            await UserJoinedVoiceChannel(user);
         else
-            UserLeftVoiceChannel(user, channel);
+            await UserLeftVoiceChannel(user, channel);
     }
 
     private bool ShouldTrackVoiceChannel(SocketVoiceChannel channel)
@@ -516,32 +514,31 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
     private bool UserParticipatingInVoiceChannel(SocketGuildUser user)
         => !user.IsDeafened && !user.IsMuted && !user.IsSelfDeafened && !user.IsSelfMuted;
 
-    private void UserJoinedVoiceChannel(SocketGuildUser user)
+    private TypedKey<long> GetVoiceXpKey(ulong userId)
+        => new($"xp:vc_join:{userId}");
+    
+    private async Task UserJoinedVoiceChannel(SocketGuildUser user)
     {
-        var key = $"{_creds.RedisKey()}_user_xp_vc_join_{user.Id}";
         var value = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
 
-        _cache.Redis.GetDatabase()
-              .StringSet(key,
-                  value,
-                  TimeSpan.FromMinutes(_xpConfig.Data.VoiceMaxMinutes),
-                  when: When.NotExists);
+        await _c.AddAsync(GetVoiceXpKey(user.Id),
+            value,
+            TimeSpan.FromMinutes(_xpConfig.Data.VoiceMaxMinutes),
+            overwrite: false);
     }
 
-    private void UserLeftVoiceChannel(SocketGuildUser user, SocketVoiceChannel channel)
+    private async Task UserLeftVoiceChannel(SocketGuildUser user, SocketVoiceChannel channel)
     {
-        var key = $"{_creds.RedisKey()}_user_xp_vc_join_{user.Id}";
-        var value = _cache.Redis.GetDatabase().StringGet(key);
-        _cache.Redis.GetDatabase().KeyDelete(key);
+        var key = GetVoiceXpKey(user.Id);
+        var result = await _c.GetAsync(key);
+        if (!await _c.RemoveAsync(key))
+            return;
 
         // Allow for if this function gets called multiple times when a user leaves a channel.
-        if (value.IsNull)
+        if (!result.TryGetValue(out var unixTime))
             return;
 
-        if (!value.TryParse(out long startUnixTime))
-            return;
-
-        var dateStart = DateTimeOffset.FromUnixTimeSeconds(startUnixTime);
+        var dateStart = DateTimeOffset.FromUnixTimeSeconds(unixTime);
         var dateEnd = DateTimeOffset.UtcNow;
         var minutes = (dateEnd - dateStart).TotalMinutes;
         var xp = _xpConfig.Data.VoiceXpPerMinute * minutes;
@@ -577,7 +574,7 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
         if (arg.Author is not SocketGuildUser user || user.IsBot)
             return Task.CompletedTask;
 
-        _ = Task.Run(() =>
+        _ = Task.Run(async () =>
         {
             if (!ShouldTrackXp(user, arg.Channel.Id))
                 return;
@@ -593,7 +590,7 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
             if (xp <= 0)
                 return;
 
-            if (!SetUserRewarded(user.Id))
+            if (!await SetUserRewardedAsync(user.Id))
                 return;
 
             _addMessageXp.Enqueue(new()
@@ -650,16 +647,14 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
         return Enumerable.Empty<ulong>();
     }
 
-    private bool SetUserRewarded(ulong userId)
-    {
-        var r = _cache.Redis.GetDatabase();
-        var key = $"{_creds.RedisKey()}_user_xp_gain_{userId}";
+    private static TypedKey<bool> GetUserRewKey(ulong userId)
+        => new($"xp:user_gain:{userId}");
 
-        return r.StringSet(key,
+    private async Task<bool> SetUserRewardedAsync(ulong userId)
+        => await _c.AddAsync(GetUserRewKey(userId),
             true,
-            TimeSpan.FromMinutes(_xpConfig.Data.MessageXpCooldown),
-            when: When.NotExists);
-    }
+            expiry: TimeSpan.FromMinutes(_xpConfig.Data.MessageXpCooldown),
+            overwrite: false);
 
     public async Task<FullUserStats> GetUserStatsAsync(IGuildUser user)
     {
@@ -782,7 +777,7 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
                 }
             }.WithFallbackFonts(_fonts.FallBackFonts);
 
-            using var img = Image.Load<Rgba32>(_images.XpBackground, out var imageFormat);
+            using var img = Image.Load<Rgba32>(await GetXpBackgroundAsync(stats.User.UserId), out var imageFormat);
             if (template.User.Name.Show)
             {
                 var fontSize = (int)(template.User.Name.FontSize * 0.9);
@@ -979,8 +974,8 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
                 {
                     var avatarUrl = stats.User.RealAvatarUrl();
 
-                    var (succ, data) = await _cache.TryGetImageDataAsync(avatarUrl);
-                    if (!succ)
+                    var result = await _c.GetImageDataAsync(avatarUrl);
+                    if (!result.TryPickT0(out var data, out _))
                     {
                         using (var http = _httpFactory.CreateClient())
                         {
@@ -999,7 +994,7 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
                             }
                         }
 
-                        await _cache.SetImageDataAsync(avatarUrl, data);
+                        await _c.SetImageDataAsync(avatarUrl, data);
                     }
 
                     using var toDraw = Image.Load(data);
@@ -1033,7 +1028,13 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
             return output;
         });
 
-// #if GLOBAL_NADEKO
+    private async Task<byte[]> GetXpBackgroundAsync(ulong userId)
+    {
+        var img = await _images.GetXpBackgroundImageAsync();
+        return img;
+    }
+
+    // #if GLOBAL_NADEKO
     private async Task DrawFrame(Image<Rgba32> img, ulong userId)
     {
         var patron = await _ps.GetPatronAsync(userId);
@@ -1103,8 +1104,8 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
             try
             {
                 var imgUrl = new Uri(stats.User.Club.ImageUrl);
-                var (succ, data) = await _cache.TryGetImageDataAsync(imgUrl);
-                if (!succ)
+                var result = await _c.GetImageDataAsync(imgUrl);
+                if (!result.TryPickT0(out var data, out _))
                 {
                     using (var http = _httpFactory.CreateClient())
                     using (var temp = await http.GetAsync(imgUrl, HttpCompletionOption.ResponseHeadersRead))
@@ -1127,7 +1128,7 @@ public class XpService : INService, IReadyExecutor, IExecNoCommand
                         }
                     }
 
-                    await _cache.SetImageDataAsync(imgUrl, data);
+                    await _c.SetImageDataAsync(imgUrl, data);
                 }
 
                 using var toDraw = Image.Load(data);

src/NadekoBot/NadekoBot.csproj

@@ -39,6 +39,7 @@
             <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
         </PackageReference>
         <PackageReference Include="Html2Markdown" Version="5.0.2.561" />
+        <PackageReference Include="Microsoft.CodeAnalysis.CSharp.Scripting" Version="4.2.0" />
         
         <PackageReference Include="Microsoft.Extensions.Configuration" Version="6.0.1" />
         <PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="6.0.1" />
@@ -52,6 +53,7 @@
         <PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="6.0.0" />
         <PackageReference Include="Microsoft.SyndicationFeed.ReaderWriter" Version="1.0.2" />
         <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
+        <PackageReference Include="NonBlocking" Version="2.0.0" />
         <PackageReference Include="OneOf" Version="3.0.216" />
         <PackageReference Include="Scrutor" Version="4.1.0" />
         <PackageReference Include="Serilog.Sinks.Console" Version="4.0.1" />
@@ -69,15 +71,15 @@
         <PackageReference Include="JetBrains.Annotations" Version="2022.1.0" />
         
         <!-- Db-related packages -->
-        <PackageReference Include="Microsoft.EntityFrameworkCore" Version="6.0.5" />
+        <PackageReference Include="Microsoft.EntityFrameworkCore" Version="6.0.6" />
         <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="6.0.5">
             <PrivateAssets>all</PrivateAssets>
             <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
         </PackageReference>
         
-        <PackageReference Include="linq2db.EntityFrameworkCore" Version="6.7.1" />
+        <PackageReference Include="linq2db.EntityFrameworkCore" Version="6.8.0" />
         
-        <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="6.0.5" />        
+        <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="6.0.6" />        
         <PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="6.0.4" />
         <PackageReference Include="Pomelo.EntityFrameworkCore.MySql" Version="6.0.1" />
         

src/NadekoBot/Services/IDataCache.cs

@@ -1,35 +0,0 @@
-#nullable disable
-using StackExchange.Redis;
-
-namespace NadekoBot.Services;
-
-public interface IDataCache
-{
-    ConnectionMultiplexer Redis { get; }
-    IImageCache LocalImages { get; }
-    ILocalDataCache LocalData { get; }
-
-    Task<(bool Success, byte[] Data)> TryGetImageDataAsync(Uri key);
-    Task<(bool Success, string Data)> TryGetAnimeDataAsync(string key);
-    Task<(bool Success, string Data)> TryGetNovelDataAsync(string key);
-    Task SetImageDataAsync(Uri key, byte[] data);
-    Task SetAnimeDataAsync(string link, string data);
-    Task SetNovelDataAsync(string link, string data);
-    TimeSpan? AddTimelyClaim(ulong id, int period);
-    TimeSpan? TryAddRatelimit(ulong id, string name, int expireIn);
-    void RemoveAllTimelyClaims();
-    bool TryAddAffinityCooldown(ulong userId, out TimeSpan? time);
-    bool TryAddDivorceCooldown(ulong userId, out TimeSpan? time);
-    bool TryGetEconomy(out string data);
-    void SetEconomy(string data);
-
-    Task<TOut> GetOrAddCachedDataAsync<TParam, TOut>(
-        string key,
-        Func<TParam, Task<TOut>> factory,
-        TParam param,
-        TimeSpan expiry)
-        where TOut : class;
-
-    DateTime GetLastCurrencyDecay();
-    void SetLastCurrencyDecay();
-}

src/NadekoBot/Services/IImageCache.cs

@@ -1,29 +0,0 @@
-#nullable disable
-namespace NadekoBot.Services;
-
-public interface IImageCache
-{
-    ImageUrls ImageUrls { get; }
-
-    IReadOnlyList<byte[]> Heads { get; }
-    IReadOnlyList<byte[]> Tails { get; }
-
-    IReadOnlyList<byte[]> Dice { get; }
-
-    IReadOnlyList<byte[]> SlotEmojis { get; }
-    IReadOnlyList<byte[]> Currency { get; }
-
-    byte[] SlotBackground { get; }
-
-    byte[] RategirlMatrix { get; }
-    byte[] RategirlDot { get; }
-
-    byte[] XpBackground { get; }
-
-    byte[] Rip { get; }
-    byte[] RipOverlay { get; }
-
-    byte[] GetCard(string key);
-
-    Task Reload();
-}

src/NadekoBot/Services/ILocalDataCache.cs

@@ -6,8 +6,8 @@ namespace NadekoBot.Services;
 
 public interface ILocalDataCache
 {
-    IReadOnlyDictionary<string, SearchPokemon> Pokemons { get; }
-    IReadOnlyDictionary<string, SearchPokemonAbility> PokemonAbilities { get; }
-    IReadOnlyDictionary<int, string> PokemonMap { get; }
-    TriviaQuestion[] TriviaQuestions { get; }
+    Task<IReadOnlyDictionary<string, SearchPokemon>> GetPokemonsAsync();
+    Task<IReadOnlyDictionary<string, SearchPokemonAbility>> GetPokemonAbilitiesAsync();
+    Task<TriviaQuestionModel[]> GetTriviaQuestionsAsync();
+    Task<PokemonNameId[]> GetPokemonMapAsync();
 }

src/NadekoBot/Services/Impl/BotCredsProvider.cs

@@ -37,8 +37,15 @@ public sealed class BotCredsProvider : IBotCredsProvider
     public BotCredsProvider(int? totalShards = null)
     {
         _totalShards = totalShards;
-        if (!File.Exists(CredsExamplePath))
-            File.WriteAllText(CredsExamplePath, Yaml.Serializer.Serialize(_creds));
+        try
+        {
+            if (!File.Exists(CredsExamplePath))
+                File.WriteAllText(CredsExamplePath, Yaml.Serializer.Serialize(_creds));
+        }
+        catch
+        {
+            // this can fail in docker containers
+        }
 
         MigrateCredentials();
 
@@ -96,8 +103,10 @@ public sealed class BotCredsProvider : IBotCredsProvider
             if (string.IsNullOrWhiteSpace(_creds.RedisOptions))
                 _creds.RedisOptions = "127.0.0.1,syncTimeout=3000";
 
-            if (string.IsNullOrWhiteSpace(_creds.CoinmarketcapApiKey))
-                _creds.CoinmarketcapApiKey = "e79ec505-0913-439d-ae07-069e296a6079";
+            // replace the old generated key with the shared key
+            if (string.IsNullOrWhiteSpace(_creds.CoinmarketcapApiKey)
+                || _creds.CoinmarketcapApiKey.StartsWith("e79ec505-0913"))
+                _creds.CoinmarketcapApiKey = "3077537c-7dfb-4d97-9a60-56fc9a9f5035";
 
             _creds.TotalShards = _totalShards ?? _creds.TotalShards;
         }
@@ -163,9 +172,10 @@ public sealed class BotCredsProvider : IBotCredsProvider
         if (File.Exists(CREDS_FILE_NAME))
         {
             var creds = Yaml.Deserializer.Deserialize<Creds>(File.ReadAllText(CREDS_FILE_NAME));
-            if (creds.Version <= 4)
+            if (creds.Version <= 5)
             {
-                creds.Version = 5;
+                creds.Version = 6;
+                creds.BotCache = BotCacheImplemenation.Redis;
                 File.WriteAllText(CREDS_FILE_NAME, Yaml.Serializer.Serialize(creds));
             }
         }

src/NadekoBot/Services/Impl/LocalDataCache.cs

@@ -0,0 +1,82 @@
+using NadekoBot.Common.Pokemon;
+using NadekoBot.Modules.Games.Common.Trivia;
+using System.Text.Json;
+using System.Text.Json.Serialization;
+
+namespace NadekoBot.Services;
+
+public sealed class LocalDataCache : ILocalDataCache, INService
+{
+    private const string POKEMON_ABILITIES_FILE = "data/pokemon/pokemon_abilities.json";
+    private const string POKEMON_LIST_FILE = "data/pokemon/pokemon_list.json";
+    private const string POKEMON_MAP_PATH = "data/pokemon/name-id_map.json";
+    private const string QUESTIONS_FILE = "data/trivia_questions.json";
+
+    private readonly IBotCache _cache;
+
+    private readonly JsonSerializerOptions _opts = new JsonSerializerOptions()
+    {
+        AllowTrailingCommas = true,
+        NumberHandling = JsonNumberHandling.AllowReadingFromString,
+        PropertyNameCaseInsensitive = true
+    };
+
+    public LocalDataCache(IBotCache cache)
+        => _cache = cache;
+
+    private async Task<T?> GetOrCreateCachedDataAsync<T>(
+        TypedKey<T> key,
+        string fileName)
+        => await _cache.GetOrAddAsync(key,
+            async () =>
+            {
+                if (!File.Exists(fileName))
+                {
+                    Log.Warning($"{fileName} is missing. Relevant data can't be loaded");
+                    return default;
+                }
+
+                try
+                {
+                    await using var stream = File.OpenRead(fileName);
+                    return await JsonSerializer.DeserializeAsync<T>(stream, _opts);
+                }
+                catch (Exception ex)
+                {
+                    Log.Error(ex,
+                        "Error reading {FileName} file: {ErrorMessage}",
+                        fileName,
+                        ex.Message);
+
+                    return default;
+                }
+            });
+
+
+    private static TypedKey<IReadOnlyDictionary<string, SearchPokemon>> _pokemonListKey
+        = new("pokemon:list");
+
+    public async Task<IReadOnlyDictionary<string, SearchPokemon>?> GetPokemonsAsync()
+        => await GetOrCreateCachedDataAsync(_pokemonListKey, POKEMON_LIST_FILE);
+
+
+    private static TypedKey<IReadOnlyDictionary<string, SearchPokemonAbility>> _pokemonAbilitiesKey
+        = new("pokemon:abilities");
+
+    public async Task<IReadOnlyDictionary<string, SearchPokemonAbility>?> GetPokemonAbilitiesAsync()
+        => await GetOrCreateCachedDataAsync(_pokemonAbilitiesKey, POKEMON_ABILITIES_FILE);
+
+
+    private static TypedKey<PokemonNameId[]> _pokeMapKey
+        = new("pokemon:ab_map");
+
+    public async Task<PokemonNameId[]?> GetPokemonMapAsync()
+        => await GetOrCreateCachedDataAsync(_pokeMapKey, POKEMON_MAP_PATH);
+
+
+    private static TypedKey<TriviaQuestionModel[]> _triviaKey
+        = new("trivia:questions");
+
+    public async Task<TriviaQuestionModel[]?> GetTriviaQuestionsAsync()
+        => await GetOrCreateCachedDataAsync(_triviaKey, QUESTIONS_FILE);
+}