Added quick and dirty escape to asterisks.

Booster counter

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,26 @@
+---
+name: Bug report
+about: Create an issue to report a bug
+title: ''
+labels: bug
+assignees: bobloy
+
+---
+
+**Describe the bug**
+<!--A clear and concise description of what the bug is.-->
+
+**To Reproduce**
+<!--Steps to reproduce the behavior:-->
+1. Load cog '...'
+2. Run command '....'
+3. See error
+
+**Expected behavior**
+<!--A clear and concise description of what you expected to happen.-->
+
+**Screenshots or Error Messages**
+<!--If applicable, add screenshots to help explain your problem.-->
+
+**Additional context**
+<!--Add any other context about the problem here.-->

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,14 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: "[Feature Request]"
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+<!--A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]-->
+
+**Describe the solution you'd like**
+<!--A clear and concise description of what you want to happen. Include which cog or cogs this would interact with-->

.github/ISSUE_TEMPLATE/new-audiotrivia-list.md

@@ -0,0 +1,26 @@
+---
+name: New AudioTrivia List
+about: Submit a new AudioTrivia list to be added
+title: "[AudioTrivia Submission]"
+labels: 'cog: audiotrivia'
+assignees: bobloy
+
+---
+
+**What is this trivia list?**
+<!--What's in the list? What kind of category is?-->
+
+**Number of Questions**
+<!--Rough estimate at the number of question in this list-->
+
+**Original Content?**
+<!--Did you come up with this list yourself or did you get it from some else's work?-->
+<!--If no, be sure to include the source-->
+- [ ] Yes
+- [ ] No
+
+
+**Did I test the list?**
+<!--Did you already try out the list and find no bugs?-->
+- [ ] Yes
+- [ ] No

.github/labeler.yml

@@ -0,0 +1,62 @@
+'cog: announcedaily':
+  - announcedaily/*
+'cog: audiotrivia':
+  - audiotrivia/*
+'cog: ccrole':
+  - ccrole/*
+'cog: chatter':
+  - chatter/*
+'cog: conquest':
+  - conquest/*
+'cog: dad':
+  - dad/*
+'cog: exclusiverole':
+  - exclusiverole/*
+'cog: fifo':
+  - fifo/*
+'cog: firstmessage':
+  - firstmessage/*
+'cog: flag':
+  - flag/*
+'cog: forcemention':
+  - forcemention/*
+'cog: hangman':
+  - hangman
+'cog: infochannel':
+  - infochannel/*
+'cog: isitdown':
+  - isitdown/*
+'cog: launchlib':
+  - launchlib/*
+'cog: leaver':
+  - leaver/*
+'cog: lovecalculator':
+  - lovecalculator/*
+'cog: lseen':
+  - lseen/*
+'cog: nudity':
+  - nudity/*
+'cog: planttycoon':
+  - planttycoon/*
+'cog: qrinvite':
+  - qrinvite/*
+'cog: reactrestrict':
+  - reactrestrict/*
+'cog: recyclingplant':
+  - recyclingplant/*
+'cog: rpsls':
+  - rpsls/*
+'cog: sayurl':
+  - sayurl/*
+'cog: scp':
+  - scp/*
+'cog: stealemoji':
+  - stealemoji/*
+'cog: timerole':
+  - timerole/*
+'cog: tts':
+  - tts/*
+'cog: unicode':
+  - unicode/*
+'cog: werewolf':
+  - werewolf/*

.github/workflows/black_check.yml

@@ -0,0 +1,20 @@
+# GitHub Action that uses Black to reformat the Python code in an incoming pull request.
+# If all Python code in the pull request is compliant with Black then this Action does nothing.
+# Othewrwise, Black is run and its changes are committed back to the incoming pull request.
+# https://github.com/cclauss/autoblack
+
+name: black
+on: [pull_request]
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v2
+        with:
+          python-version: '3.8'
+      - name: Install Black
+        run: pip install --upgrade --no-cache-dir black
+      - name: Run black --check .
+        run: black --check --diff -l 99 .

.github/workflows/labeler.yml

@@ -0,0 +1,19 @@
+# This workflow will triage pull requests and apply a label based on the
+# paths that are modified in the pull request.
+#
+# To use this workflow, you will need to set up a .github/labeler.yml
+# file with configuration.  For more information, see:
+# https://github.com/actions/labeler
+
+name: Labeler
+on: [pull_request_target]
+
+jobs:
+  label:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/labeler@2.2.0
+      with:
+        repo-token: "${{ secrets.GITHUB_TOKEN }}"

.gitignore

@@ -3,3 +3,5 @@
 venv/
 v-data/
 database.sqlite3
+/venv3.4/
+/.venv/

README.md

@@ -9,18 +9,23 @@ Cog Function
 | ccrole | **Release** | <details><summary>Create custom commands that also assign roles</summary>May have some bugs, please create an issue if you find any</details> |
 | chatter | **Beta** | <details><summary>Chat-bot trained to talk like your guild</summary>Missing some key features, but currently functional. See [Chatter](https://github.com/bobloy/Fox-V3/tree/master/chatter) for install instructions</details> |
 | coglint | **Alpha** | <details><summary>Error check code in python syntax posted to discord</summary>Works, but probably needs more turning to work for cogs</details> |
+| conquest | **Alpha** | <details><summary>Manage maps for war games and RPGs</summary>Lots of additional features are planned, currently function with simple map</details> |
 | dad | **Beta** | <details><summary>Tell dad jokes</summary>Works great!</details> |
 | exclusiverole | **Alpha** | <details><summary>Prevent certain roles from getting any other roles</summary>Fully functional, but pretty simple</details> |
+| fifo | **Alpha** | <details><summary>Schedule commands to be run at certain times or intervals</summary>Just released, please report bugs as you find them. Only works for bot owner for now</details> |
 | fight | **Incomplete** | <details><summary>Organize bracket tournaments within discord</summary>Still in-progress, a massive project</details> |
+| firstmessage | **Release** | <details><summary>Simple cog to provide a jump link to the first message in a channel/summary>Just released, please report bugs as you find them.</details> |
 | flag | **Alpha** | <details><summary>Create temporary marks on users that expire after specified time</summary>Ported, will not import old data. Please report bugs</details> |
 | forcemention | **Alpha** | <details><summary>Mentions unmentionable roles</summary>Very simple cog, mention doesn't persist</details> |
 | hangman | **Beta** | <details><summary>Play a game of hangman</summary>Some visual glitches and needs more customization</details> |
 | howdoi | **Incomplete** | <details><summary>Ask coding questions and get results from StackExchange</summary>Not yet functional</details> |
+| infochannel | **Beta** | <details><summary>Create a channel to display server info</summary>Due to rate limits, this does not update as often as it once did</details> |
+| isitdown | **Beta** | <details><summary>Check if a website/url is down</summary>Just released, please report bugs</details> |
+| launchlib | **Beta** | <details><summary>Access rocket launch data</summary>Just released, please report bugs</details> |
 | leaver | **Beta** | <details><summary>Send a message in a channel when a user leaves the server</summary>Seems to be functional, please report any bugs or suggestions</details> |
-| infochannel | **Beta** | <details><summary>Create a channel to display server info</summary>Just released, please report bugs</details> |
 | lovecalculator | **Alpha** | <details><summary>Calculate the love between two users</summary>[Snap-Ons] Just updated to V3</details> |
 | lseen | **Alpha** | <details><summary>Track when a member was last online</summary>Alpha release, please report bugs</details> |
-| nudity | **Incomplete** | <details><summary>Checks for NSFW images posted in non-NSFW channels</summary>Library this is based on has a bug, waiting for author to merge my PR</details> |
+| nudity | **Alpha** | <details><summary>Checks for NSFW images posted in non-NSFW channels</summary>Switched libraries, now functional</details> |
 | planttycoon | **Alpha** | <details><summary>Grow your own plants!</summary>[Snap-Ons] Updated to V3, likely to contain bugs</details> |
 | qrinvite | **Alpha** | <details><summary>Create a QR code invite for the server</summary>Alpha release, please report any bugs</details> |
 | reactrestrict | **Alpha** | <details><summary>Removes reactions by role per channel</summary>A bit clunky, but functional</details> |
@@ -35,7 +40,7 @@ Cog Function
 | unicode | **Alpha** | <details><summary>Encode and Decode unicode characters</summary>[Snap-Ons] Just updated to V3</details> |
 | werewolf | **Pre-Alpha** | <details><summary>Play the classic party game Werewolf within discord</summary>Another massive project currently being developed, will be fully customizable</details> |
 
-Check out my V2 cogs at [Fox-Cogs v2](https://github.com/bobloy/Fox-Cogs)
+Check out *Deprecated* my V2 cogs at [Fox-Cogs v2](https://github.com/bobloy/Fox-Cogs)
 
 # Installation
 ### Recommended - Built-in Downloader
@@ -48,7 +53,7 @@ Check out my V2 cogs at [Fox-Cogs v2](https://github.com/bobloy/Fox-Cogs)
 # Contact
 Get support on the [Third Party Cog Server](https://discord.gg/GET4DVk)
 
-Feel free to @ me in the #support_othercogs channel
+Feel free to @ me in the #support_fox-v3 channel
 
 Discord: Bobloy#6513
 

announcedaily/announcedaily.py

@@ -38,6 +38,10 @@ class AnnounceDaily(Cog):
         self.config.register_global(**default_global)
         self.config.register_guild(**default_guild)
 
+    async def red_delete_data_for_user(self, **kwargs):
+        """Nothing to delete"""
+        return
+
     async def _get_msgs(self):
         return DEFAULT_MESSAGES + await self.config.messages()
 
@@ -50,8 +54,7 @@ class AnnounceDaily(Cog):
 
         Do `[p]help annd <subcommand>` for more details
         """
-        if ctx.invoked_subcommand is None:
+        pass
-            pass
 
     @commands.command()
     @checks.guildowner()

announcedaily/info.json

@@ -2,15 +2,12 @@
   "author": [
     "Bobloy"
   ],
-  "bot_version": [
+  "min_bot_version": "3.3.0",
-    3,
-    0,
-    0
-  ],
   "description": "Send daily announcements to all servers at a specified times",
-  "hidden": true,
+  "hidden": false,
   "install_msg": "Thank you for installing AnnounceDaily! Get started with `[p]load announcedaily` and `[p]help AnnounceDaily`",
   "short": "Send daily announcements",
+  "end_user_data_statement": "This cog does not store any End User Data",
   "tags": [
     "bobloy"
   ]

audiotrivia/audiosession.py

@@ -1,21 +1,25 @@
 """Module to manage audio trivia sessions."""
 import asyncio
+import logging
 
-import lavalink
 from redbot.cogs.trivia import TriviaSession
+from redbot.cogs.trivia.session import _parse_answers
+from redbot.core.utils.chat_formatting import bold
+
+log = logging.getLogger("red.fox_v3.audiotrivia.audiosession")
 
 
 class AudioSession(TriviaSession):
     """Class to run a session of audio trivia"""
 
-    def __init__(self, ctx, question_list: dict, settings: dict, player: lavalink.Player):
+    def __init__(self, ctx, question_list: dict, settings: dict, audio=None):
         super().__init__(ctx, question_list, settings)
 
-        self.player = player
+        self.audio = audio
 
     @classmethod
-    def start(cls, ctx, question_list, settings, player: lavalink.Player = None):
+    def start(cls, ctx, question_list, settings, audio=None):
-        session = cls(ctx, question_list, settings, player)
+        session = cls(ctx, question_list, settings, audio)
         loop = ctx.bot.loop
         session._task = loop.create_task(session.run())
         return session
@@ -23,52 +27,95 @@ class AudioSession(TriviaSession):
     async def run(self):
         """Run the audio trivia session.
 
-                In order for the trivia session to be stopped correctly, this should
+        In order for the trivia session to be stopped correctly, this should
-                only be called internally by `TriviaSession.start`.
+        only be called internally by `TriviaSession.start`.
-                """
+        """
         await self._send_startup_msg()
         max_score = self.settings["max_score"]
         delay = self.settings["delay"]
+        audio_delay = self.settings["audio_delay"]
         timeout = self.settings["timeout"]
-        for question, answers in self._iter_questions():
+        if self.audio is not None:
+            import lavalink
+
+            player = lavalink.get_player(self.ctx.guild.id)
+            player.store("channel", self.ctx.channel.id)  # What's this for? I dunno
+            await self.audio.set_player_settings(self.ctx)
+        else:
+            lavalink = None
+            player = False
+
+        for question, answers, audio_url in self._iter_questions():
             async with self.ctx.typing():
                 await asyncio.sleep(3)
             self.count += 1
-            await self.player.stop()
+            msg = bold(f"Question number {self.count}!") + f"\n\n{question}"
-
+            if player:
-            msg = "**Question number {}!**\n\nName this audio!".format(self.count)
+                await player.stop()
-            await self.ctx.send(msg)
+            if audio_url:
-            # print("Audio question: {}".format(question))
+                if not player:
-
+                    log.debug("Got an audio question in a non-audio trivia session")
-            # await self.ctx.invoke(self.audio.play(ctx=self.ctx, query=question))
+                    continue
-            # ctx_copy = copy(self.ctx)
 
-            # await self.ctx.invoke(self.player.play, query=question)
+                load_result = await player.load_tracks(audio_url)
-            query = question.strip("<>")
+                if (
-            tracks = await self.player.get_tracks(query)
+                    load_result.has_error
-            seconds = tracks[0].length / 1000
+                    or load_result.load_type != lavalink.enums.LoadType.TRACK_LOADED
+                ):
+                    await self.ctx.maybe_send_embed(
+                        "Audio Track has an error, skipping. See logs for details"
+                    )
+                    log.info(f"Track has error: {load_result.exception_message}")
+                    continue
+                tracks = load_result.tracks
+                track = tracks[0]
+                seconds = track.length / 1000
+                track.uri = ""  # Hide the info from `now`
+                if self.settings["repeat"] and seconds < audio_delay:
+                    # Append it until it's longer than the delay
+                    tot_length = seconds + 0
+                    while tot_length < audio_delay:
+                        player.add(self.ctx.author, track)
+                        tot_length += seconds
+                else:
+                    player.add(self.ctx.author, track)
 
-            if self.settings["repeat"] and seconds < delay:
+                if not player.current:
-                tot_length = seconds + 0
+                    await player.play()
-                while tot_length < delay:
+            await self.ctx.maybe_send_embed(msg)
-                    self.player.add(self.ctx.author, tracks[0])
+            log.debug(f"Audio question: {question}")
-                    tot_length += seconds
-            else:
-                self.player.add(self.ctx.author, tracks[0])
-
-            if not self.player.current:
-                await self.player.play()
 
-            continue_ = await self.wait_for_answer(answers, delay, timeout)
+            continue_ = await self.wait_for_answer(
+                answers, audio_delay if audio_url else delay, timeout
+            )
             if continue_ is False:
                 break
             if any(score >= max_score for score in self.scores.values()):
                 await self.end_game()
                 break
         else:
-            await self.ctx.send("There are no more questions!")
+            await self.ctx.maybe_send_embed("There are no more questions!")
             await self.end_game()
 
     async def end_game(self):
         await super().end_game()
-        await self.player.disconnect()
+        if self.audio is not None:
+            await self.ctx.invoke(self.audio.command_disconnect)
+
+    def _iter_questions(self):
+        """Iterate over questions and answers for this session.
+
+        Yields
+        ------
+        `tuple`
+            A tuple containing the question (`str`) and the answers (`tuple` of
+            `str`).
+
+        """
+        for question, q_data in self.question_list:
+            answers = _parse_answers(q_data["answers"])
+            _audio = q_data["audio"]
+            if _audio:
+                yield _audio, answers, question.strip("<>")
+            else:
+                yield question, answers, _audio

audiotrivia/audiotrivia.py

@@ -1,23 +1,24 @@
 import datetime
+import logging
 import pathlib
-from typing import List
+from typing import List, Optional
 
+import discord
 import lavalink
 import yaml
 from redbot.cogs.audio import Audio
-from redbot.cogs.audio.core.utilities import validation
+from redbot.cogs.trivia.trivia import InvalidListError, Trivia, get_core_lists
-from redbot.cogs.trivia import LOG
+from redbot.core import Config, checks, commands
-from redbot.cogs.trivia.trivia import InvalidListError, Trivia
-from redbot.core import commands, Config, checks
 from redbot.core.bot import Red
 from redbot.core.data_manager import cog_data_path
-from redbot.core.utils.chat_formatting import box
+from redbot.core.utils.chat_formatting import bold, box
-# from redbot.cogs.audio.utils import userlimit
-
 
 from .audiosession import AudioSession
 
 
+log = logging.getLogger("red.fox_v3.audiotrivia")
+
+
 class AudioTrivia(Trivia):
     """
     Upgrade to the Trivia cog that enables audio trivia
@@ -27,12 +28,11 @@ class AudioTrivia(Trivia):
     def __init__(self, bot: Red):
         super().__init__()
         self.bot = bot
-        self.audio = None
         self.audioconf = Config.get_conf(
             self, identifier=651171001051118411410511810597, force_registration=True
         )
 
-        self.audioconf.register_guild(delay=30.0, repeat=True)
+        self.audioconf.register_guild(audio_delay=30.0, repeat=True)
 
     @commands.group()
     @commands.guild_only()
@@ -43,122 +43,112 @@ class AudioTrivia(Trivia):
         settings_dict = await audioset.all()
         msg = box(
             "**Audio settings**\n"
-            "Answer time limit: {delay} seconds\n"
+            "Answer time limit: {audio_delay} seconds\n"
             "Repeat Short Audio: {repeat}"
             "".format(**settings_dict),
             lang="py",
         )
         await ctx.send(msg)
 
-    @atriviaset.command(name="delay")
+    @atriviaset.command(name="timelimit")
-    async def atriviaset_delay(self, ctx: commands.Context, seconds: float):
+    async def atriviaset_timelimit(self, ctx: commands.Context, seconds: float):
         """Set the maximum seconds permitted to answer a question."""
         if seconds < 4.0:
             await ctx.send("Must be at least 4 seconds.")
             return
         settings = self.audioconf.guild(ctx.guild)
-        await settings.delay.set(seconds)
+        await settings.audo_delay.set(seconds)
-        await ctx.send("Done. Maximum seconds to answer set to {}.".format(seconds))
+        await ctx.maybe_send_embed(f"Done. Maximum seconds to answer set to {seconds}.")
 
     @atriviaset.command(name="repeat")
     async def atriviaset_repeat(self, ctx: commands.Context, true_or_false: bool):
         """Set whether or not short audio will be repeated"""
         settings = self.audioconf.guild(ctx.guild)
         await settings.repeat.set(true_or_false)
-        await ctx.send("Done. Repeating short audio is now set to {}.".format(true_or_false))
+        await ctx.maybe_send_embed(f"Done. Repeating short audio is now set to {true_or_false}.")
 
     @commands.group(invoke_without_command=True)
     @commands.guild_only()
     async def audiotrivia(self, ctx: commands.Context, *categories: str):
-        """Start trivia session on the specified category.
+        """Start trivia session on the specified category or categories.
 
-                You may list multiple categories, in which case the trivia will involve
+        Includes Audio categories.
-                questions from all of them.
+        You may list multiple categories, in which case the trivia will involve
-                """
+        questions from all of them.
+        """
         if not categories and ctx.invoked_subcommand is None:
             await ctx.send_help()
             return
-
-        if self.audio is None:
-            self.audio: Audio = self.bot.get_cog("Audio")
-
-        if self.audio is None:
-            await ctx.send("Audio is not loaded. Load it and try again")
-            return
-
         categories = [c.lower() for c in categories]
         session = self._get_trivia_session(ctx.channel)
         if session is not None:
-            await ctx.send("There is already an ongoing trivia session in this channel.")
+            await ctx.maybe_send_embed(
-            return
+                "There is already an ongoing trivia session in this channel."
-        status = await self.audio.config.status()
-        notify = await self.audio.config.guild(ctx.guild).notify()
-
-        if status:
-            await ctx.send(
-                "It is recommended to disable audio status with `{}audioset status`".format(ctx.prefix)
-            )
-
-        if notify:
-            await ctx.send(
-                "It is recommended to disable audio notify with `{}audioset notify`".format(ctx.prefix)
-            )
-
-        if not self.audio._player_check(ctx):
-            try:
-                if not ctx.author.voice.channel.permissions_for(
-                    ctx.me
-                ).connect or self.audio.is_vc_full(ctx.author.voice.channel):
-                    return await ctx.send("I don't have permission to connect to your channel.")
-                await lavalink.connect(ctx.author.voice.channel)
-                lavaplayer = lavalink.get_player(ctx.guild.id)
-                lavaplayer.store("connect", datetime.datetime.utcnow())
-            except AttributeError:
-                return await ctx.send("Connect to a voice channel first.")
-
-        lavaplayer = lavalink.get_player(ctx.guild.id)
-        lavaplayer.store("channel", ctx.channel.id)  # What's this for? I dunno
-
-        await self.audio.set_player_settings(ctx)
-
-        if not ctx.author.voice or ctx.author.voice.channel != lavaplayer.channel:
-            return await ctx.send(
-                "You must be in the voice channel to use the audiotrivia command."
             )
-
+            return
         trivia_dict = {}
         authors = []
+        any_audio = False
         for category in reversed(categories):
             # We reverse the categories so that the first list's config takes
             # priority over the others.
             try:
                 dict_ = self.get_audio_list(category)
             except FileNotFoundError:
-                await ctx.send(
+                await ctx.maybe_send_embed(
-                    "Invalid category `{0}`. See `{1}audiotrivia list`"
+                    f"Invalid category `{category}`. See `{ctx.prefix}audiotrivia list`"
                     " for a list of trivia categories."
-                    "".format(category, ctx.prefix)
                 )
             except InvalidListError:
-                await ctx.send(
+                await ctx.maybe_send_embed(
                     "There was an error parsing the trivia list for"
-                    " the `{}` category. It may be formatted"
+                    f" the `{category}` category. It may be formatted"
-                    " incorrectly.".format(category)
+                    " incorrectly."
                 )
             else:
-                trivia_dict.update(dict_)
+                is_audio = dict_.pop("AUDIO", False)
-                authors.append(trivia_dict.pop("AUTHOR", None))
+                authors.append(dict_.pop("AUTHOR", None))
+                trivia_dict.update(
+                    {_q: {"audio": is_audio, "answers": _a} for _q, _a in dict_.items()}
+                )
+                any_audio = any_audio or is_audio
                 continue
             return
         if not trivia_dict:
-            await ctx.send(
+            await ctx.maybe_send_embed(
                 "The trivia list was parsed successfully, however it appears to be empty!"
             )
             return
 
+        if not any_audio:
+            audio = None
+        else:
+            audio: Optional["Audio"] = self.bot.get_cog("Audio")
+            if audio is None:
+                await ctx.send("Audio lists were parsed but Audio is not loaded!")
+                return
+            status = await audio.config.status()
+            notify = await audio.config.guild(ctx.guild).notify()
+
+            if status:
+                await ctx.maybe_send_embed(
+                    f"It is recommended to disable audio status with `{ctx.prefix}audioset status`"
+                )
+
+            if notify:
+                await ctx.maybe_send_embed(
+                    f"It is recommended to disable audio notify with `{ctx.prefix}audioset notify`"
+                )
+
+            failed = await ctx.invoke(audio.command_summon)
+            if failed:
+                return
+            lavaplayer = lavalink.get_player(ctx.guild.id)
+            lavaplayer.store("channel", ctx.channel.id)  # What's this for? I dunno
+
         settings = await self.config.guild(ctx.guild).all()
         audiosettings = await self.audioconf.guild(ctx.guild).all()
-        config = trivia_dict.pop("CONFIG", None)
+        config = trivia_dict.pop("CONFIG", {"answer": None})["answer"]
         if config and settings["allow_override"]:
             settings.update(config)
         settings["lists"] = dict(zip(categories, reversed(authors)))
@@ -166,22 +156,33 @@ class AudioTrivia(Trivia):
         # Delay in audiosettings overwrites delay in settings
         combined_settings = {**settings, **audiosettings}
         session = AudioSession.start(
-            ctx=ctx, question_list=trivia_dict, settings=combined_settings, player=lavaplayer
+            ctx,
+            trivia_dict,
+            combined_settings,
+            audio,
         )
         self.trivia_sessions.append(session)
-        LOG.debug("New audio trivia session; #%s in %d", ctx.channel, ctx.guild.id)
+        log.debug("New audio trivia session; #%s in %d", ctx.channel, ctx.guild.id)
 
     @audiotrivia.command(name="list")
     @commands.guild_only()
     async def audiotrivia_list(self, ctx: commands.Context):
-        """List available trivia categories."""
+        """List available trivia including audio categories."""
-        lists = set(p.stem for p in self._audio_lists())
+        lists = {p.stem for p in self._all_audio_lists()}
-
+        if await ctx.embed_requested():
-        msg = box("**Available trivia lists**\n\n{}".format(", ".join(sorted(lists))))
+            await ctx.send(
-        if len(msg) > 1000:
+                embed=discord.Embed(
-            await ctx.author.send(msg)
+                    title="Available trivia lists",
-            return
+                    colour=await ctx.embed_colour(),
-        await ctx.send(msg)
+                    description=", ".join(sorted(lists)),
+                )
+            )
+        else:
+            msg = box(bold("Available trivia lists") + "\n\n" + ", ".join(sorted(lists)))
+            if len(msg) > 1000:
+                await ctx.author.send(msg)
+            else:
+                await ctx.send(msg)
 
     def get_audio_list(self, category: str) -> dict:
         """Get the audiotrivia list corresponding to the given category.
@@ -198,7 +199,7 @@ class AudioTrivia(Trivia):
 
         """
         try:
-            path = next(p for p in self._audio_lists() if p.stem == category)
+            path = next(p for p in self._all_audio_lists() if p.stem == category)
         except StopIteration:
             raise FileNotFoundError("Could not find the `{}` category.".format(category))
 
@@ -210,13 +211,15 @@ class AudioTrivia(Trivia):
             else:
                 return dict_
 
-    def _audio_lists(self) -> List[pathlib.Path]:
+    def _all_audio_lists(self) -> List[pathlib.Path]:
+        # Custom trivia lists uploaded with audiotrivia. Not necessarily audio lists
         personal_lists = [p.resolve() for p in cog_data_path(self).glob("*.yaml")]
 
-        return personal_lists + get_core_lists()
+        # Add to that custom lists uploaded with trivia and core lists
+        return personal_lists + get_core_audio_lists() + self._all_lists()
 
 
-def get_core_lists() -> List[pathlib.Path]:
+def get_core_audio_lists() -> List[pathlib.Path]:
     """Return a list of paths for all trivia lists packaged with the bot."""
     core_lists_path = pathlib.Path(__file__).parent.resolve() / "data/lists"
     return list(core_lists_path.glob("*.yaml"))

audiotrivia/data/lists/audioanime.yaml

@@ -1,4 +1,5 @@
 AUTHOR: Plab
+AUDIO: "[Audio] Identify this Anime!"
 https://www.youtube.com/watch?v=2uq34TeWEdQ:
 - 'Hagane no Renkinjutsushi (2009)'
 - '(2009) الخيميائي المعدني الكامل'

audiotrivia/data/lists/audionhlgoalhorns.yaml

@@ -1,4 +1,5 @@
 AUTHOR: Lazar
+AUDIO: "[Audio] Identify this NHL Team by their goal horn"
 https://youtu.be/6OejNXrGkK0:
 - Anaheim Ducks
 - Anaheim

audiotrivia/data/lists/non_funcitonal_lists/games.yaml

@@ -1,13 +1,14 @@
 AUTHOR: Plab
-https://www.youtube.com/watch?v=--bWm9hhoZo:
+NEEDS: New links for all songs.
+https://www.youtube.com/watch?v=f9O2Rjn1azc:
 - Transistor
-https://www.youtube.com/watch?v=-4nCbgayZNE:
+https://www.youtube.com/watch?v=PgUhYFkVdSY:
 - Dark Cloud 2
 - Dark Cloud II
-https://www.youtube.com/watch?v=-64NlME4lJU:
+https://www.youtube.com/watch?v=1T1RZttyMwU:
 - Mega Man 7
 - Mega Man VII
-https://www.youtube.com/watch?v=-AesqnudNuw:
+https://www.youtube.com/watch?v=AdDbbzuq1vY:
 - Mega Man 9
 - Mega Man IX
 https://www.youtube.com/watch?v=-BmGDtP2t7M:

audiotrivia/info.json

@@ -2,15 +2,12 @@
   "author": [
     "Bobloy"
   ],
-  "bot_version": [
+  "min_bot_version": "3.3.0",
-    3,
-    0,
-    0
-  ],
   "description": "Start an Audio Trivia game",
   "hidden": false,
   "install_msg": "Thank you for installing Audio trivia!\n You **MUST** unload trivia to use this (`[p]unload trivia`)\n Then you can get started with `[p]load audiotrivia` and `[p]help AudioTrivia`",
   "short": "Start an Audio Trivia game",
+  "end_user_data_statement": "This cog expands the core Audio and Trivia cogs without collecting any additional End User Data.\nSee the core End User Data storage for more information",
   "tags": [
     "fox",
     "bobloy",

ccrole/ccrole.py

@@ -1,11 +1,51 @@
 import asyncio
+import logging
 import re
 
 import discord
+from discord.ext.commands import RoleConverter, Greedy, CommandError, ArgumentParsingError
 from discord.ext.commands.view import StringView
 from redbot.core import Config, checks, commands
 from redbot.core.bot import Red
 from redbot.core.utils.chat_formatting import box, pagify
+from redbot.core.utils.mod import get_audit_reason
+
+log = logging.getLogger("red.fox_v3.ccrole")
+
+
+async def _get_roles_from_content(ctx, content):
+    # greedy = Greedy[RoleConverter]
+    view = StringView(content)
+    rc = RoleConverter()
+
+    # "Borrowed" from discord.ext.commands.Command._transform_greedy_pos
+    result = []
+    while not view.eof:
+        # for use with a manual undo
+        previous = view.index
+
+        view.skip_ws()
+        try:
+            argument = view.get_quoted_word()
+            value = await rc.convert(ctx, argument)
+        except (CommandError, ArgumentParsingError):
+            view.index = previous
+            break
+        else:
+            result.append(value)
+
+    return [r.id for r in result]
+
+    # Old method
+    # content_list = content.split(",")
+    # try:
+    #     role_list = [
+    #         discord.utils.get(ctx.guild.roles, name=role.strip(" ")).id for role in content_list
+    #     ]
+    # except (discord.HTTPException, AttributeError):  # None.id is attribute error
+    #     return None
+    # else:
+    #     return role_list
 
 
 class CCRole(commands.Cog):
@@ -22,14 +62,17 @@ class CCRole(commands.Cog):
 
         self.config.register_guild(**default_guild)
 
+    async def red_delete_data_for_user(self, **kwargs):
+        """Nothing to delete"""
+        return
+
     @commands.guild_only()
     @commands.group()
     async def ccrole(self, ctx: commands.Context):
         """Custom commands management with roles
 
         Highly customizable custom commands with role management."""
-        if not ctx.invoked_subcommand:
+        pass
-            await ctx.send_help()
 
     @ccrole.command(name="add")
     @checks.mod_or_permissions(administrator=True)
@@ -38,6 +81,12 @@ class CCRole(commands.Cog):
 
         When adding text, put arguments in `{}` to eval them
         Options: `{author}`, `{target}`, `{server}`, `{channel}`, `{message}`"""
+
+        # TODO: Clean this up so it's not so repetitive
+        # The call/answer format has better options as well
+        # Saying "none" over and over can trigger automod actions as well
+        # Also, allow `ctx.tick()` instead of sending a message
+
         command = command.lower()
         if command in self.bot.all_commands:
             await ctx.send("That command is already a standard command.")
@@ -59,7 +108,8 @@ class CCRole(commands.Cog):
 
         # Roles to add
         await ctx.send(
-            "What roles should it add? (Must be **comma separated**)\nSay `None` to skip adding roles"
+            "What roles should it add?\n"
+            "Say `None` to skip adding roles"
         )
 
         def check(m):
@@ -73,14 +123,15 @@ class CCRole(commands.Cog):
 
         arole_list = []
         if answer.content.upper() != "NONE":
-            arole_list = await self._get_roles_from_content(ctx, answer.content)
+            arole_list = await _get_roles_from_content(ctx, answer.content)
             if arole_list is None:
                 await ctx.send("Invalid answer, canceling")
                 return
 
         # Roles to remove
         await ctx.send(
-            "What roles should it remove? (Must be comma separated)\nSay `None` to skip removing roles"
+            "What roles should it remove?\n"
+            "Say `None` to skip removing roles"
         )
         try:
             answer = await self.bot.wait_for("message", timeout=120, check=check)
@@ -90,14 +141,15 @@ class CCRole(commands.Cog):
 
         rrole_list = []
         if answer.content.upper() != "NONE":
-            rrole_list = await self._get_roles_from_content(ctx, answer.content)
+            rrole_list = await _get_roles_from_content(ctx, answer.content)
             if rrole_list is None:
                 await ctx.send("Invalid answer, canceling")
                 return
 
         # Roles to use
         await ctx.send(
-            "What roles are allowed to use this command? (Must be comma separated)\nSay `None` to allow all roles"
+            "What roles are allowed to use this command?\n"
+            "Say `None` to allow all roles"
         )
 
         try:
@@ -108,13 +160,15 @@ class CCRole(commands.Cog):
 
         prole_list = []
         if answer.content.upper() != "NONE":
-            prole_list = await self._get_roles_from_content(ctx, answer.content)
+            prole_list = await _get_roles_from_content(ctx, answer.content)
             if prole_list is None:
                 await ctx.send("Invalid answer, canceling")
                 return
 
         # Selfrole
-        await ctx.send("Is this a targeted command?(yes/no)\nNo will make this a selfrole command")
+        await ctx.send(
+            "Is this a targeted command?(yes/no)\n" "No will make this a selfrole command"
+        )
 
         try:
             answer = await self.bot.wait_for("message", timeout=120, check=check)
@@ -132,7 +186,7 @@ class CCRole(commands.Cog):
         # Message to send
         await ctx.send(
             "What message should the bot say when using this command?\n"
-            "Say `None` to send the default `Success!` message\n"
+            "Say `None` to send no message and just react with ✅\n"
             "Eval Options: `{author}`, `{target}`, `{server}`, `{channel}`, `{message}`\n"
             "For example: `Welcome {target.mention} to {server.name}!`"
         )
@@ -143,7 +197,7 @@ class CCRole(commands.Cog):
             await ctx.send("Timed out, canceling")
             return
 
-        text = "Success!"
+        text = None
         if answer.content.upper() != "NONE":
             text = answer.content
 
@@ -176,7 +230,7 @@ class CCRole(commands.Cog):
             await self.config.guild(guild).cmdlist.set_raw(command, value=None)
             await ctx.send("Custom command successfully deleted.")
 
-    @ccrole.command(name="details")
+    @ccrole.command(name="details", aliases=["detail"])
     async def ccrole_details(self, ctx, command: str):
         """Provide details about passed custom command"""
         guild = ctx.guild
@@ -197,13 +251,13 @@ class CCRole(commands.Cog):
             if not role_list:
                 return "None"
             return ", ".join(
-                [discord.utils.get(ctx.guild.roles, id=roleid).name for roleid in role_list]
+                discord.utils.get(ctx.guild.roles, id=roleid).name for roleid in role_list
             )
 
-        embed.add_field(name="Text", value="```{}```".format(cmd["text"]))
+        embed.add_field(name="Text", value="```{}```".format(cmd["text"]), inline=False)
-        embed.add_field(name="Adds Roles", value=process_roles(cmd["aroles"]), inline=True)
+        embed.add_field(name="Adds Roles", value=process_roles(cmd["aroles"]), inline=False)
-        embed.add_field(name="Removes Roles", value=process_roles(cmd["rroles"]), inline=True)
+        embed.add_field(name="Removes Roles", value=process_roles(cmd["rroles"]), inline=False)
-        embed.add_field(name="Role Restrictions", value=process_roles(cmd["proles"]), inline=True)
+        embed.add_field(name="Role Restrictions", value=process_roles(cmd["proles"]), inline=False)
 
         await ctx.send(embed=embed)
 
@@ -221,7 +275,7 @@ class CCRole(commands.Cog):
             )
             return
 
-        cmd_list = ", ".join([ctx.prefix + c for c in sorted(cmd_list.keys())])
+        cmd_list = ", ".join(ctx.prefix + c for c in sorted(cmd_list.keys()))
         cmd_list = "Custom commands:\n\n" + cmd_list
 
         if (
@@ -240,14 +294,17 @@ class CCRole(commands.Cog):
         https://github.com/Cog-Creators/Red-DiscordBot/blob/V3/develop/redbot/cogs/customcom/customcom.py#L508
         for the message filtering
         """
+        # This covers message.author.bot check
+        if not await self.bot.message_eligible_as_command(message):
+            return
+
         ###########
         is_private = isinstance(message.channel, discord.abc.PrivateChannel)
 
-        # user_allowed check, will be replaced with self.bot.user_allowed or
+        if is_private or len(message.content) < 2:
-        # something similar once it's added
+            return
-        user_allowed = True
 
-        if len(message.content) < 2 or is_private or not user_allowed or message.author.bot:
+        if await self.bot.cog_disabled_in_guild(self, message.guild):
             return
 
         ctx = await self.bot.get_context(message)
@@ -258,50 +315,18 @@ class CCRole(commands.Cog):
         # Thank you Cog-Creators
 
         cmd = ctx.invoked_with
-        cmd = cmd.lower()  # Continues the proud case_insentivity tradition of ccrole
+        cmd = cmd.lower()  # Continues the proud case-insensitivity tradition of ccrole
         guild = ctx.guild
         # message = ctx.message  # Unneeded since switch to `on_message_without_command` from `on_command_error`
 
-        cmdlist = self.config.guild(guild).cmdlist
+        cmd_list = self.config.guild(guild).cmdlist
         # cmd = message.content[len(prefix) :].split()[0].lower()
-        cmd = await cmdlist.get_raw(cmd, default=None)
+        cmd = await cmd_list.get_raw(cmd, default=None)
 
         if cmd is not None:
             await self.eval_cc(cmd, message, ctx)
-
-    # @commands.Cog.listener()
-    # async def on_message(self, message: discord.Message):
-    #     if len(message.content) < 2 or message.guild is None:
-    #         return
-    #
-    #     ctx: commands.Context = await self.bot.get_context(message)
-    #     cmd = ctx.invoked_with
-    #     guild = message.guild
-    #     # try:
-    #     #     prefix = await self.get_prefix(message)
-    #     # except ValueError:
-    #     #     return
-    #
-    #     # prefix = ctx.prefix
-    #
-    #     cmdlist = self.config.guild(guild).cmdlist
-    #     # cmd = message.content[len(prefix) :].split()[0].lower()
-    #     cmd = await cmdlist.get_raw(cmd, default=None)
-    #
-    #     if cmd is not None:
-    #         await self.eval_cc(cmd, message, ctx)
-
-    async def _get_roles_from_content(self, ctx, content):
-        content_list = content.split(",")
-        try:
-            role_list = [
-                discord.utils.get(ctx.guild.roles, name=role.strip(" ")).id
-                for role in content_list
-            ]
-        except (discord.HTTPException, AttributeError):  # None.id is attribute error
-            return None
         else:
-            return role_list
+            log.debug(f"No custom command named {ctx.invoked_with} found")
 
     async def get_prefix(self, message: discord.Message) -> str:
         """
@@ -321,23 +346,13 @@ class CCRole(commands.Cog):
                 return p
         raise ValueError
 
-    async def eval_cc(self, cmd, message, ctx):
+    async def eval_cc(self, cmd, message: discord.Message, ctx: commands.Context):
         """Does all the work"""
-        if cmd["proles"] and not (
+        if cmd["proles"] and not {role.id for role in message.author.roles} & set(cmd["proles"]):
-            set(role.id for role in message.author.roles) & set(cmd["proles"])
+            log.debug(f"{message.author} missing required role to execute {ctx.invoked_with}")
-        ):
             return  # Not authorized, do nothing
 
         if cmd["targeted"]:
-            # try:
-            #     arg1 = message.content.split(maxsplit=1)[1]
-            # except IndexError:  # .split() return list of len<2
-            #     target = None
-            # else:
-            #     target = discord.utils.get(
-            #         message.guild.members, mention=arg1
-            #     )
-
             view: StringView = ctx.view
             view.skip_ws()
 
@@ -356,47 +371,43 @@ class CCRole(commands.Cog):
             else:
                 target = None
 
-            # try:
-            #     arg1 = ctx.args[1]
-            # except IndexError:  # args is list of len<2
-            #     target = None
-            # else:
-            #     target = discord.utils.get(
-            #         message.guild.members, mention=arg1
-            #     )
-
             if not target:
-                out_message = "This custom command is targeted! @mention a target\n`{} <target>`".format(
+                out_message = (
-                    ctx.invoked_with
+                    f"This custom command is targeted! @mention a target\n`"
+                    f"{ctx.invoked_with} <target>`"
                 )
-                await message.channel.send(out_message)
+                await ctx.send(out_message)
                 return
         else:
             target = message.author
 
+        reason = get_audit_reason(message.author)
+
         if cmd["aroles"]:
             arole_list = [
                 discord.utils.get(message.guild.roles, id=roleid) for roleid in cmd["aroles"]
             ]
-            # await self.bot.send_message(message.channel, "Adding: "+str([str(arole) for arole in arole_list]))
             try:
-                await target.add_roles(*arole_list)
+                await target.add_roles(*arole_list, reason=reason)
             except discord.Forbidden:
-                await message.channel.send("Permission error: Unable to add roles")
+                log.exception(f"Permission error: Unable to add roles")
-        await asyncio.sleep(1)
+                await ctx.send("Permission error: Unable to add roles")
 
         if cmd["rroles"]:
             rrole_list = [
                 discord.utils.get(message.guild.roles, id=roleid) for roleid in cmd["rroles"]
             ]
-            # await self.bot.send_message(message.channel, "Removing: "+str([str(rrole) for rrole in rrole_list]))
             try:
-                await target.remove_roles(*rrole_list)
+                await target.remove_roles(*rrole_list, reason=reason)
             except discord.Forbidden:
-                await message.channel.send("Permission error: Unable to remove roles")
+                log.exception(f"Permission error: Unable to remove roles")
+                await ctx.send("Permission error: Unable to remove roles")
 
-        out_message = self.format_cc(cmd, message, target)
+        if cmd["text"] is not None:
-        await message.channel.send(out_message)
+            out_message = self.format_cc(cmd, message, target)
+            await ctx.send(out_message, allowed_mentions=discord.AllowedMentions())
+        else:
+            await ctx.tick()
 
     def format_cc(self, cmd, message, target):
         out = cmd["text"]
@@ -410,6 +421,7 @@ class CCRole(commands.Cog):
         """
         For security reasons only specific objects are allowed
         Internals are ignored
+        Copied from customcom.CustomCommands.transform_parameter and added `target`
         """
         raw_result = "{" + result + "}"
         objects = {

ccrole/info.json

@@ -2,15 +2,12 @@
   "author": [
     "Bobloy"
   ],
-  "bot_version": [
+  "min_bot_version": "3.4.0",
-    3,
+  "description": "Creates custom commands to adjust roles and send custom messages",
-    0,
-    0
-  ],
-  "description": "[Incomplete] Creates custom commands to adjust roles and send custom messages",
   "hidden": false,
   "install_msg": "Thank you for installing Custom Commands w/ Roles. Get started with `[p]load ccrole` and `[p]help CCRole`",
-  "short": "[Incomplete] Creates commands that adjust roles",
+  "short": "Creates commands that adjust roles",
+  "end_user_data_statement": "This cog does not store any End User Data",
   "tags": [
     "fox",
     "bobloy",

chatter/README.md

@@ -29,7 +29,7 @@ Chatter by default uses spaCy's `en_core_web_md` training model, which is ~50 MB
 
 Chatter can potential use spaCy's `en_core_web_lg` training model, which is ~800 MB
 
-Chatter uses as sqlite database that can potentially take up a large amount os disk space,
+Chatter uses as sqlite database that can potentially take up a large amount of disk space,
 depending on how much training Chatter has done. 
 
 The sqlite database can be safely deleted at any time. Deletion will only erase training data.
@@ -50,68 +50,59 @@ Linux is a bit easier, but only tested on Debian and Ubuntu.
 
 ## Windows Prerequisites
 
-Install these on your windows machine before attempting the installation
+**Requires 64 Bit Python to continue on Windows.**
+
+Install these on your windows machine before attempting the installation:
 
 [Visual Studio C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
 
 [Pandoc - Universal Document Converter](https://pandoc.org/installing.html)
 
 ## Methods
-### Windows - Manually
+### Automatic
-#### Step 1: Built-in Downloader
+
+This method requires some luck to pull off.
 
-You need to get a copy of the requirements.txt provided with chatter, I recommend this method.
+#### Step 1: Add repo and install cog
 
 ```
 [p]repo add Fox https://github.com/bobloy/Fox-V3
+[p]cog install Fox chatter
 ```
 
-#### Step 2: Install Requirements
+If you get an error at this step, stop and skip to one of the manual methods below.
 
-Make sure you have your virtual environment that you installed Red on activated before starting this step. See the Red Docs for details on how.
+#### Step 2: Install additional dependencies
 
-In a terminal running as an admin, navigate to the directory containing this repo. 
+Here you need to decide which training models you want to have available to you.
 
-I've used my install directory as an example.
+Shutdown the bot and run any number of these in the console:
 
 ```
-cd C:\Users\Bobloy\AppData\Local\Red-DiscordBot\Red-DiscordBot\data\bobbot\cogs\RepoManager\repos\Fox\chatter
+python -m spacy download en_core_web_sm  # ~15 MB
-pip install -r requirements.txt
-pip install --no-deps "chatterbot>=1.1"
-```
-
-#### Step 3: Load Chatter
-
-```
-[p]cog install Fox chatter
-[p]load chatter
-```
 
-### Linux - Manually
+python -m spacy download en_core_web_md  # ~50 MB
 
-#### Step 1: Built-in Downloader
+python -m spacy download en_core_web_lg  # ~750 MB (CPU Optimized)
 
+python -m spacy download en_core_web_trf  # ~500 MB (GPU Optimized)
 ```
-[p]cog install Chatter
-```
-
-#### Step 2: Install Requirements
 
-In your console with your virtual environment activated:
+#### Step 3: Load the cog and get started
 
 ```
-pip install --no-deps "chatterbot>=1.1"
+[p]load chatter
 ```
 
-### Step 3: Load Chatter
+### Windows - Manually
+Deprecated
 
-```
+### Linux - Manually
-[p]load chatter
+Deprecated
-```
 
 # Configuration
 
-Chatter works out the the box without any training by learning as it goes, 
+Chatter works out the box without any training by learning as it goes, 
 but will have very poor and repetitive responses at first.
 
 Initial training is recommended to speed up its learning.
@@ -162,12 +153,53 @@ This command trains Chatter on the specified channel based on the configured
 settings. This can take a long time to process.
 
 
+### Train Ubuntu
+
+```
+[p]chatter trainubuntu
+``` 
+*WARNING:* This will trigger a large download and use a lot of processing power
+
+This command trains Chatter on the publicly available Ubuntu Dialogue Corpus. (It'll talk like a geek)
+
+
 ## Switching Algorithms
 
 ```
 [p]chatter algorithm X
 ```
+or
+```
+[p]chatter algo X 0.95
+```
 
 Chatter can be configured to use one of three different Similarity algorithms.
 
 Changing this can help if the response speed is too slow, but can reduce the accuracy of results.
+
+The second argument is the maximum similarity threshold,
+lowering that will make the bot stop searching sooner.
+
+Default maximum similarity threshold is 0.90
+
+
+## Switching Pretrained Models
+
+```
+[p]chatter model X
+```
+
+Chatter can be configured to use one of three pretrained statistical models for English.
+
+I have not noticed any advantage to changing this, 
+but supposedly it would help by splitting the search term into more useful parts.
+
+See [here](https://spacy.io/models) for more info on spaCy models.
+
+Before you're able to use the *large* model (option 3), you must install it through pip.
+
+*Warning:* This is ~800MB download.
+
+```
+[p]pipinstall https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-2.3.1/en_core_web_lg-2.3.1.tar.gz#egg=en_core_web_lg
+``` 

chatter/__init__.py

@@ -1,8 +1,10 @@
 from .chat import Chatter
 
 
-def setup(bot):
+async def setup(bot):
-    bot.add_cog(Chatter(bot))
+    cog = Chatter(bot)
+    await cog.initialize()
+    bot.add_cog(cog)
 
 
 # __all__ = (

chatter/chat.py

@@ -1,19 +1,42 @@
 import asyncio
+import logging
 import os
 import pathlib
+from collections import defaultdict
 from datetime import datetime, timedelta
+from functools import partial
+from typing import Dict, List, Optional
 
 import discord
 from chatterbot import ChatBot
 from chatterbot.comparisons import JaccardSimilarity, LevenshteinDistance, SpacySimilarity
 from chatterbot.response_selection import get_random_response
-from chatterbot.trainers import ChatterBotCorpusTrainer, ListTrainer
+from chatterbot.trainers import ChatterBotCorpusTrainer, ListTrainer, UbuntuCorpusTrainer
-from redbot.core import Config, commands
+from redbot.core import Config, checks, commands
 from redbot.core.commands import Cog
 from redbot.core.data_manager import cog_data_path
+from redbot.core.utils.predicates import MessagePredicate
 
+from chatter.trainers import MovieTrainer, TwitterCorpusTrainer, UbuntuCorpusTrainer2
 
-class ENG_LG:  # TODO: Add option to use this large model
+chatterbot_log = logging.getLogger("red.fox_v3.chatterbot")
+log = logging.getLogger("red.fox_v3.chatter")
+
+
+def my_local_get_prefix(prefixes, content):
+    for p in prefixes:
+        if content.startswith(p):
+            return p
+    return None
+
+
+class ENG_TRF:
+    ISO_639_1 = "en_core_web_trf"
+    ISO_639 = "eng"
+    ENGLISH_NAME = "English"
+
+
+class ENG_LG:
     ISO_639_1 = "en_core_web_lg"
     ISO_639 = "eng"
     ENGLISH_NAME = "English"
@@ -25,45 +48,88 @@ class ENG_MD:
     ENGLISH_NAME = "English"
 
 
+class ENG_SM:
+    ISO_639_1 = "en_core_web_sm"
+    ISO_639 = "eng"
+    ENGLISH_NAME = "English"
+
+
 class Chatter(Cog):
     """
     This cog trains a chatbot that will talk like members of your Guild
     """
 
+    models = [ENG_SM, ENG_MD, ENG_LG, ENG_TRF]
+    algos = [SpacySimilarity, JaccardSimilarity, LevenshteinDistance]
+
     def __init__(self, bot):
         super().__init__()
         self.bot = bot
         self.config = Config.get_conf(self, identifier=6710497116116101114)
-        default_global = {}
+        default_global = {"learning": True, "model_number": 0, "algo_number": 0, "threshold": 0.90}
-        default_guild = {"whitelist": None, "days": 1, "convo_delta": 15}
+        self.default_guild = {
+            "whitelist": None,
+            "days": 1,
+            "convo_delta": 15,
+            "chatchannel": None,
+            "reply": True,
+        }
         path: pathlib.Path = cog_data_path(self)
         self.data_path = path / "database.sqlite3"
 
-        self.chatbot = self._create_chatbot(self.data_path, SpacySimilarity, 0.45, ENG_MD)
+        # TODO: Move training_model and similarity_algo to config
+        # TODO: Add an option to see current settings
+
+        self.tagger_language = ENG_SM
+        self.similarity_algo = SpacySimilarity
+        self.similarity_threshold = 0.90
+        self.chatbot = None
         # self.chatbot.set_trainer(ListTrainer)
 
         # self.trainer = ListTrainer(self.chatbot)
 
         self.config.register_global(**default_global)
-        self.config.register_guild(**default_guild)
+        self.config.register_guild(**self.default_guild)
 
         self.loop = asyncio.get_event_loop()
 
-    def _create_chatbot(
+        self._guild_cache = defaultdict(dict)
-            self, data_path, similarity_algorithm, similarity_threshold, tagger_language
+        self._global_cache = {}
-    ):
+
+        self._last_message_per_channel: Dict[Optional[discord.Message]] = defaultdict(lambda: None)
+
+    async def red_delete_data_for_user(self, **kwargs):
+        """Nothing to delete"""
+        return
+
+    async def initialize(self):
+        all_config = dict(self.config.defaults["GLOBAL"])
+        all_config.update(await self.config.all())
+        model_number = all_config["model_number"]
+        algo_number = all_config["algo_number"]
+        threshold = all_config["threshold"]
+
+        self.tagger_language = self.models[model_number]
+        self.similarity_algo = self.algos[algo_number]
+        self.similarity_threshold = threshold
+        self.chatbot = self._create_chatbot()
+
+    def _create_chatbot(self):
+
         return ChatBot(
             "ChatterBot",
-            storage_adapter="chatterbot.storage.SQLStorageAdapter",
+            # storage_adapter="chatterbot.storage.SQLStorageAdapter",
-            database_uri="sqlite:///" + str(data_path),
+            storage_adapter="chatter.storage_adapters.MyDumbSQLStorageAdapter",
-            statement_comparison_function=similarity_algorithm,
+            database_uri="sqlite:///" + str(self.data_path),
+            statement_comparison_function=self.similarity_algo,
             response_selection_method=get_random_response,
             logic_adapters=["chatterbot.logic.BestMatch"],
-            # maximum_similarity_threshold=similarity_threshold,
+            maximum_similarity_threshold=self.similarity_threshold,
-            tagger_language=tagger_language,
+            tagger_language=self.tagger_language,
+            logger=chatterbot_log,
         )
 
-    async def _get_conversation(self, ctx, in_channel: discord.TextChannel = None):
+    async def _get_conversation(self, ctx, in_channels: List[discord.TextChannel]):
         """
         Compiles all conversation in the Guild this bot can get it's hands on
         Currently takes a stupid long time
@@ -77,21 +143,13 @@ class Chatter(Cog):
             return msg.clean_content
 
         def new_conversation(msg, sent, out_in, delta):
-            # if sent is None:
+            # Should always be positive numbers
-            #     return False
-
-            # Don't do "too short" processing here. Sometimes people don't respond.
-            # if len(out_in) < 2:
-            #     return False
-
-            # print(msg.created_at - sent)
-
             return msg.created_at - sent >= delta
 
-        for channel in ctx.guild.text_channels:
+        for channel in in_channels:
-            if in_channel:
+            # if in_channel:
-                channel = in_channel
+            #     channel = in_channel
-            await ctx.send("Gathering {}".format(channel.mention))
+            await ctx.maybe_send_embed("Gathering {}".format(channel.mention))
             user = None
             i = 0
             send_time = after - timedelta(days=100)  # Makes the first message a new message
@@ -99,7 +157,7 @@ class Chatter(Cog):
             try:
 
                 async for message in channel.history(
-                        limit=None, after=after, oldest_first=True
+                    limit=None, after=after, oldest_first=True
                 ).filter(
                     predicate=predicate
                 ):  # type: discord.Message
@@ -125,11 +183,47 @@ class Chatter(Cog):
             except discord.HTTPException:
                 pass
 
-            if in_channel:
+            # if in_channel:
-                break
+            #     break
 
         return out
 
+    def _train_twitter(self, *args, **kwargs):
+        trainer = TwitterCorpusTrainer(self.chatbot)
+        trainer.train(*args, **kwargs)
+        return True
+
+    def _train_ubuntu(self):
+        trainer = UbuntuCorpusTrainer(
+            self.chatbot, ubuntu_corpus_data_directory=cog_data_path(self) / "ubuntu_data"
+        )
+        trainer.train()
+        return True
+
+    async def _train_movies(self):
+        trainer = MovieTrainer(self.chatbot, cog_data_path(self))
+        return await trainer.asynctrain()
+
+    async def _train_ubuntu2(self, intensity):
+        train_kwarg = {}
+        if intensity == 196:
+            train_kwarg["train_dialogue"] = False
+            train_kwarg["train_196"] = True
+        elif intensity == 301:
+            train_kwarg["train_dialogue"] = False
+            train_kwarg["train_301"] = True
+        elif intensity == 497:
+            train_kwarg["train_dialogue"] = False
+            train_kwarg["train_196"] = True
+            train_kwarg["train_301"] = True
+        elif intensity >= 9000:  # NOT 9000!
+            train_kwarg["train_dialogue"] = True
+            train_kwarg["train_196"] = True
+            train_kwarg["train_301"] = True
+
+        trainer = UbuntuCorpusTrainer2(self.chatbot, cog_data_path(self))
+        return await trainer.asynctrain(**train_kwarg)
+
     def _train_english(self):
         trainer = ChatterBotCorpusTrainer(self.chatbot)
         # try:
@@ -141,13 +235,10 @@ class Chatter(Cog):
     def _train(self, data):
         trainer = ListTrainer(self.chatbot)
         total = len(data)
-        # try:
         for c, convo in enumerate(data, 1):
+            log.info(f"{c} / {total}")
             if len(convo) > 1:  # TODO: Toggleable skipping short conversations
-                print(f"{c} / {total}")
                 trainer.train(convo)
-        # except:
-        #     return False
         return True
 
     @commands.group(invoke_without_command=False)
@@ -155,19 +246,82 @@ class Chatter(Cog):
         """
         Base command for this cog. Check help for the commands list.
         """
-        if ctx.invoked_subcommand is None:
+        self._guild_cache[ctx.guild.id] = {}  # Clear cache when modifying values
-            pass
+        self._global_cache = {}
 
+    @commands.admin()
+    @chatter.command(name="channel")
+    async def chatter_channel(
+        self, ctx: commands.Context, channel: Optional[discord.TextChannel] = None
+    ):
+        """
+        Set a channel that the bot will respond in without mentioning it
+
+        Pass with no channel object to clear this guild's channel
+        """
+        if channel is None:
+            await self.config.guild(ctx.guild).chatchannel.set(None)
+            await ctx.maybe_send_embed("Chat channel for guild is cleared")
+        else:
+            if channel.guild != ctx.guild:
+                await ctx.maybe_send_embed("What are you trying to pull here? :eyes:")
+                return
+            await self.config.guild(ctx.guild).chatchannel.set(channel.id)
+            await ctx.maybe_send_embed(f"Chat channel is now {channel.mention}")
+
+    @commands.admin()
+    @chatter.command(name="reply")
+    async def chatter_reply(self, ctx: commands.Context, toggle: Optional[bool] = None):
+        """
+        Toggle bot reply to messages if conversation continuity is not present
+
+        """
+        reply = await self.config.guild(ctx.guild).reply()
+        if toggle is None:
+            toggle = not reply
+        await self.config.guild(ctx.guild).reply.set(toggle)
+
+        if toggle:
+            await ctx.maybe_send_embed(
+                "I will now respond to you if conversation continuity is not present"
+            )
+        else:
+            await ctx.maybe_send_embed(
+                "I will not reply to your message if conversation continuity is not present, anymore"
+            )
+
+    @commands.is_owner()
+    @chatter.command(name="learning")
+    async def chatter_learning(self, ctx: commands.Context, toggle: Optional[bool] = None):
+        """
+        Toggle the bot learning from its conversations.
+
+        This is a global setting.
+        This is on by default.
+        """
+        learning = await self.config.learning()
+        if toggle is None:
+            toggle = not learning
+        await self.config.learning.set(toggle)
+
+        if toggle:
+            await ctx.maybe_send_embed("I will now learn from conversations.")
+        else:
+            await ctx.maybe_send_embed("I will no longer learn from conversations.")
+
+    @commands.is_owner()
     @chatter.command(name="cleardata")
     async def chatter_cleardata(self, ctx: commands.Context, confirm: bool = False):
         """
-        This command will erase all training data and reset your configuration settings
+        This command will erase all training data and reset your configuration settings.
 
-        Use `[p]chatter cleardata True`
+        This applies to all guilds.
+
+        Use `[p]chatter cleardata True` to confirm.
         """
 
         if not confirm:
-            await ctx.send(
+            await ctx.maybe_send_embed(
                 "Warning, this command will erase all your training data and reset your configuration\n"
                 "If you want to proceed, run the command again as `[p]chatter cleardata True`"
             )
@@ -182,35 +336,93 @@ class Chatter(Cog):
                 try:
                     os.remove(self.data_path)
                 except PermissionError:
-                    await ctx.maybe_send_embed("Failed to clear training database. Please wait a bit and try again")
+                    await ctx.maybe_send_embed(
+                        "Failed to clear training database. Please wait a bit and try again"
+                    )
 
-            self._create_chatbot(self.data_path, SpacySimilarity, 0.45, ENG_MD)
+            self._create_chatbot()
 
         await ctx.tick()
 
-    @chatter.command(name="algorithm")
+    @commands.is_owner()
-    async def chatter_algorithm(self, ctx: commands.Context, algo_number: int):
+    @chatter.command(name="algorithm", aliases=["algo"])
+    async def chatter_algorithm(
+        self, ctx: commands.Context, algo_number: int, threshold: float = None
+    ):
         """
-        Switch the active logic algorithm to one of the three. Default after reload is Spacy
+        Switch the active logic algorithm to one of the three. Default is Spacy
 
         0: Spacy
         1: Jaccard
         2: Levenshtein
         """
+        if algo_number < 0 or algo_number > 2:
+            await ctx.send_help()
+            return
 
-        algos = [(SpacySimilarity, 0.45), (JaccardSimilarity, 0.75), (LevenshteinDistance, 0.75)]
+        if threshold is not None:
+            if threshold >= 1 or threshold <= 0:
+                await ctx.maybe_send_embed(
+                    "Threshold must be a number between 0 and 1 (exclusive)"
+                )
+                return
+            else:
+                self.similarity_threshold = threshold
+                await self.config.threshold.set(self.similarity_threshold)
 
-        if algo_number < 0 or algo_number > 2:
+        self.similarity_algo = self.algos[algo_number]
+        await self.config.algo_number.set(algo_number)
+
+        async with ctx.typing():
+            self.chatbot = self._create_chatbot()
+
+            await ctx.tick()
+
+    @commands.is_owner()
+    @chatter.command(name="model")
+    async def chatter_model(self, ctx: commands.Context, model_number: int):
+        """
+        Switch the active model to one of the three. Default is Small
+
+        0: Small
+        1: Medium (Requires additional setup)
+        2: Large (Requires additional setup)
+        3. Accurate (Requires additional setup)
+        """
+        if model_number < 0 or model_number > 3:
             await ctx.send_help()
             return
 
-        self.chatbot = self._create_chatbot(
+        if model_number >= 0:
-            self.data_path, algos[algo_number][0], algos[algo_number][1], ENG_MD
+            await ctx.maybe_send_embed(
-        )
+                "Additional requirements needed. See guide before continuing.\n" "Continue?"
+            )
+            pred = MessagePredicate.yes_or_no(ctx)
+            try:
+                await self.bot.wait_for("message", check=pred, timeout=30)
+            except TimeoutError:
+                await ctx.send("Response timed out, please try again later.")
+                return
+            if not pred.result:
+                return
+
+        self.tagger_language = self.models[model_number]
+        await self.config.model_number.set(model_number)
+        async with ctx.typing():
+            self.chatbot = self._create_chatbot()
 
-        await ctx.tick()
+            await ctx.maybe_send_embed(
+                f"Model has been switched to {self.tagger_language.ISO_639_1}"
+            )
 
-    @chatter.command(name="minutes")
+    @commands.is_owner()
+    @chatter.group(name="trainset")
+    async def chatter_trainset(self, ctx: commands.Context):
+        """Commands for configuring training"""
+        pass
+
+    @commands.is_owner()
+    @chatter_trainset.command(name="minutes")
     async def minutes(self, ctx: commands.Context, minutes: int):
         """
         Sets the number of minutes the bot will consider a break in a conversation during training
@@ -221,11 +433,12 @@ class Chatter(Cog):
             await ctx.send_help()
             return
 
-        await self.config.guild(ctx.guild).convo_length.set(minutes)
+        await self.config.guild(ctx.guild).convo_delta.set(minutes)
 
         await ctx.tick()
 
-    @chatter.command(name="age")
+    @commands.is_owner()
+    @chatter_trainset.command(name="age")
     async def age(self, ctx: commands.Context, days: int):
         """
         Sets the number of days to look back
@@ -239,13 +452,23 @@ class Chatter(Cog):
         await self.config.guild(ctx.guild).days.set(days)
         await ctx.tick()
 
+    @commands.is_owner()
+    @chatter.command(name="kaggle")
+    async def chatter_kaggle(self, ctx: commands.Context):
+        """Register with the kaggle API to download additional datasets for training"""
+        if not await self.check_for_kaggle():
+            await ctx.maybe_send_embed(
+                "[Click here for instructions to setup the kaggle api](https://github.com/Kaggle/kaggle-api#api-credentials)"
+            )
+
+    @commands.is_owner()
     @chatter.command(name="backup")
     async def backup(self, ctx, backupname):
         """
         Backup your training data to a json for later use
         """
 
-        await ctx.send("Backing up data, this may take a while")
+        await ctx.maybe_send_embed("Backing up data, this may take a while")
 
         path: pathlib.Path = cog_data_path(self)
 
@@ -256,11 +479,96 @@ class Chatter(Cog):
         )
 
         if future:
-            await ctx.send(f"Backup successful! Look in {path} for your backup")
+            await ctx.maybe_send_embed(f"Backup successful! Look in {path} for your backup")
         else:
-            await ctx.send("Error occurred :(")
+            await ctx.maybe_send_embed("Error occurred :(")
+
+    @commands.is_owner()
+    @chatter.group(name="train")
+    async def chatter_train(self, ctx: commands.Context):
+        """Commands for training the bot"""
+        pass
+
+    @chatter_train.group(name="kaggle")
+    async def chatter_train_kaggle(self, ctx: commands.Context):
+        """
+        Base command for kaggle training sets.
+
+        See `[p]chatter kaggle` for details on how to enable this option
+        """
+        pass
+
+    @chatter_train_kaggle.command(name="ubuntu")
+    async def chatter_train_kaggle_ubuntu(
+        self, ctx: commands.Context, confirmation: bool = False, intensity=0
+    ):
+        """
+        WARNING: Large Download! Trains the bot using *NEW* Ubuntu Dialog Corpus data.
+        """
+
+        if not confirmation:
+            await ctx.maybe_send_embed(
+                "Warning: This command downloads ~800MB and is CPU intensive during training\n"
+                "If you're sure you want to continue, run `[p]chatter train kaggle ubuntu True`"
+            )
+            return
 
-    @chatter.command(name="trainenglish")
+        async with ctx.typing():
+            future = await self._train_ubuntu2(intensity)
+
+        if future:
+            await ctx.maybe_send_embed("Training successful!")
+        else:
+            await ctx.maybe_send_embed("Error occurred :(")
+
+    @chatter_train_kaggle.command(name="movies")
+    async def chatter_train_kaggle_movies(self, ctx: commands.Context, confirmation: bool = False):
+        """
+        WARNING: Language! Trains the bot using Cornell University's "Movie Dialog Corpus".
+
+        This training set contains dialog from a spread of movies with different MPAA.
+        This dialog includes racism, sexism, and any number of sensitive topics.
+
+        Use at your own risk.
+        """
+
+        if not confirmation:
+            await ctx.maybe_send_embed(
+                "Warning: This command downloads ~29MB and is CPU intensive during training\n"
+                "If you're sure you want to continue, run `[p]chatter train kaggle movies True`"
+            )
+            return
+
+        async with ctx.typing():
+            future = await self._train_movies()
+
+        if future:
+            await ctx.maybe_send_embed("Training successful!")
+        else:
+            await ctx.maybe_send_embed("Error occurred :(")
+
+    @chatter_train.command(name="ubuntu")
+    async def chatter_train_ubuntu(self, ctx: commands.Context, confirmation: bool = False):
+        """
+        WARNING: Large Download! Trains the bot using Ubuntu Dialog Corpus data.
+        """
+
+        if not confirmation:
+            await ctx.maybe_send_embed(
+                "Warning: This command downloads ~500MB and is CPU intensive during training\n"
+                "If you're sure you want to continue, run `[p]chatter train ubuntu True`"
+            )
+            return
+
+        async with ctx.typing():
+            future = await self.loop.run_in_executor(None, self._train_ubuntu)
+
+        if future:
+            await ctx.maybe_send_embed("Training successful!")
+        else:
+            await ctx.maybe_send_embed("Error occurred :(")
+
+    @chatter_train.command(name="english")
     async def chatter_train_english(self, ctx: commands.Context):
         """
         Trains the bot in english
@@ -269,30 +577,51 @@ class Chatter(Cog):
             future = await self.loop.run_in_executor(None, self._train_english)
 
         if future:
-            await ctx.send("Training successful!")
+            await ctx.maybe_send_embed("Training successful!")
         else:
-            await ctx.send("Error occurred :(")
+            await ctx.maybe_send_embed("Error occurred :(")
+
+    @chatter_train.command(name="list")
+    async def chatter_train_list(self, ctx: commands.Context):
+        """Trains the bot based on an uploaded list.
+
+        Must be a file in the format of a python list: ['prompt', 'response1', 'response2']
+        """
+        if not ctx.message.attachments:
+            await ctx.maybe_send_embed("You must upload a file when using this command")
+            return
+
+        attachment: discord.Attachment = ctx.message.attachments[0]
 
-    @chatter.command()
+        a_bytes = await attachment.read()
-    async def train(self, ctx: commands.Context, channel: discord.TextChannel):
+
+        await ctx.send("Not yet implemented")
+
+    @chatter_train.command(name="channel")
+    async def chatter_train_channel(
+        self, ctx: commands.Context, channels: commands.Greedy[discord.TextChannel]
+    ):
         """
-        Trains the bot based on language in this guild
+        Trains the bot based on language in this guild.
         """
+        if not channels:
+            await ctx.send_help()
+            return
 
-        await ctx.send(
+        await ctx.maybe_send_embed(
             "Warning: The cog may use significant RAM or CPU if trained on large data sets.\n"
             "Additionally, large sets will use more disk space to save the trained data.\n\n"
             "If you experience issues, clear your trained data and train again on a smaller scope."
         )
 
         async with ctx.typing():
-            conversation = await self._get_conversation(ctx, channel)
+            conversation = await self._get_conversation(ctx, channels)
 
         if not conversation:
-            await ctx.send("Failed to gather training data")
+            await ctx.maybe_send_embed("Failed to gather training data")
             return
 
-        await ctx.send(
+        await ctx.maybe_send_embed(
             "Gather successful! Training begins now\n"
             "(**This will take a long time, be patient. See console for progress**)"
         )
@@ -307,11 +636,11 @@ class Chatter(Cog):
             pass
 
         if future:
-            await ctx.send("Training successful!")
+            await ctx.maybe_send_embed("Training successful!")
         else:
-            await ctx.send("Error occurred :(")
+            await ctx.maybe_send_embed("Error occurred :(")
 
-    @commands.Cog.listener()
+    @Cog.listener()
     async def on_message_without_command(self, message: discord.Message):
         """
         Credit to https://github.com/Twentysix26/26-Cogs/blob/master/cleverbot/cleverbot.py
@@ -322,61 +651,101 @@ class Chatter(Cog):
         for the message filtering
         """
         ###########
-        is_private = isinstance(message.channel, discord.abc.PrivateChannel)
 
-        # user_allowed check, will be replaced with self.bot.user_allowed or
+        if len(message.content) < 2 or message.author.bot:
-        # something similar once it's added
+            return
-        user_allowed = True
+
+        guild: discord.Guild = getattr(message, "guild", None)
 
-        if len(message.content) < 2 or is_private or not user_allowed or message.author.bot:
+        if guild is None or await self.bot.cog_disabled_in_guild(self, guild):
             return
 
         ctx: commands.Context = await self.bot.get_context(message)
 
-        if ctx.prefix is not None:
+        if ctx.prefix is not None:  # Probably unnecessary, we're in on_message_without_command
             return
 
         ###########
         # Thank you Cog-Creators
+        channel: discord.TextChannel = message.channel
 
-        def my_local_get_prefix(prefixes, content):
+        if not self._guild_cache[guild.id]:
-            for p in prefixes:
+            self._guild_cache[guild.id] = await self.config.guild(guild).all()
-                if content.startswith(p):
+
-                    return p
+        is_reply = False  # this is only useful with in_response_to
-            return None
+        if (
+            message.reference is not None
+            and isinstance(message.reference.resolved, discord.Message)
+            and message.reference.resolved.author.id == self.bot.user.id
+        ):
+            is_reply = True  # this is only useful with in_response_to
+            pass  # this is a reply to the bot, good to go
+        elif guild is not None and channel.id == self._guild_cache[guild.id]["chatchannel"]:
+            pass  # good to go
+        else:
+            when_mentionables = commands.when_mentioned(self.bot, message)
 
-        when_mentionables = commands.when_mentioned(self.bot, message)
+            prefix = my_local_get_prefix(when_mentionables, message.content)
 
-        prefix = my_local_get_prefix(when_mentionables, message.content)
+            if prefix is None:
+                # print("not mentioned")
+                return
 
-        if prefix is None:
+            message.content = message.content.replace(prefix, "", 1)
-            # print("not mentioned")
-            return
 
-        author = message.author
+        text = message.clean_content
-        guild: discord.Guild = message.guild
 
-        channel: discord.TextChannel = message.channel
+        async with ctx.typing():
 
-        # if author.id != self.bot.user.id:
+            if is_reply:
-        #     if guild is None:
+                in_response_to = message.reference.resolved.content
-        #         to_strip = "@" + channel.me.display_name + " "
+            elif self._last_message_per_channel[ctx.channel.id] is not None:
-        #     else:
+                last_m: discord.Message = self._last_message_per_channel[ctx.channel.id]
-        #         to_strip = "@" + guild.me.display_name + " "
+                minutes = self._guild_cache[ctx.guild.id]["convo_delta"]
-        #     text = message.clean_content
+                if (datetime.utcnow() - last_m.created_at).seconds > minutes * 60:
-        #     if not text.startswith(to_strip):
+                    in_response_to = None
-        #         return
+                else:
-        #     text = text.replace(to_strip, "", 1)
+                    in_response_to = last_m.content
-
+            else:
-        # A bit more aggressive, could remove two mentions
+                in_response_to = None
-        # Or might not work at all, since mentionables are pre-cleaned_content
+
-        message.content = message.content.replace(prefix, "", 1)
+            # Always use generate reponse
-        text = message.clean_content
+            # Chatterbot tries to learn based on the result it comes up with, which is dumb
+            log.debug("Generating response")
+            Statement = self.chatbot.storage.get_object("statement")
+            future = await self.loop.run_in_executor(
+                None, self.chatbot.generate_response, Statement(text)
+            )
 
-        async with channel.typing():
+            if not self._global_cache:
-            future = await self.loop.run_in_executor(None, self.chatbot.get_response, text)
+                self._global_cache = await self.config.all()
+
+            if in_response_to is not None and self._global_cache["learning"]:
+                log.debug("learning response")
+                await self.loop.run_in_executor(
+                    None,
+                    partial(
+                        self.chatbot.learn_response,
+                        Statement(text),
+                        previous_statement=in_response_to,
+                    ),
+                )
+
+            replying = None
+            if (
+                "reply" not in self._guild_cache[guild.id] and self.default_guild["reply"]
+            ) or self._guild_cache[guild.id]["reply"]:
+                if message != ctx.channel.last_message:
+                    replying = message
 
             if future and str(future):
-                await channel.send(str(future))
+                self._last_message_per_channel[ctx.channel.id] = await channel.send(
+                    str(future), reference=replying
+                )
             else:
-                await channel.send(":thinking:")
+                await ctx.send(":thinking:")
+
+    async def check_for_kaggle(self):
+        """Check whether Kaggle is installed and configured properly"""
+        # TODO: This
+        return False

chatter/info.json

@@ -2,28 +2,18 @@
   "author": [
     "Bobloy"
   ],
-  "bot_version": [
+  "min_bot_version": "3.4.6",
-    3,
+  "description": "Create an offline chatbot that talks like your average member using Machine Learning. See setup instructions at https://github.com/bobloy/Fox-V3/tree/master/chatter",
-    3,
-    10
-  ],
-  "description": "Create an offline chatbot that talks like your average member using Machine Learning",
   "hidden": false,
-  "install_msg": "Thank you for installing Chatter! Get started ith `[p]load chatter` and `[p]help Chatter`",
+  "install_msg": "Thank you for installing Chatter! Please make sure you check the install instructions at https://github.com/bobloy/Fox-V3/blob/master/chatter/README.md\nAfter that, get started ith `[p]load chatter` and `[p]help Chatter`",
   "requirements": [
-    "git+git://github.com/gunthercox/chatterbot-corpus@master#egg=chatterbot_corpus",
+    "git+git://github.com/bobloy/ChatterBot@fox#egg=ChatterBot>=1.1.0.dev4",
-    "mathparse>=0.1,<0.2",
+    "kaggle",
-    "nltk>=3.2,<4.0",
+    "https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.1.0/en_core_web_sm-3.1.0.tar.gz#egg=en_core_web_sm",
-    "pint>=0.8.1",
+    "https://github.com/explosion/spacy-models/releases/download/en_core_web_md-3.1.0/en_core_web_md-3.1.0.tar.gz#egg=en_core_web_md"
-    "python-dateutil>=2.8,<2.9",
-    "pyyaml>=5.3,<5.4",
-    "sqlalchemy>=1.3,<1.4",
-    "pytz",
-    "https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz#egg=en_core_web_sm",
-    "https://github.com/explosion/spacy-models/releases/download/en_core_web_md-2.3.1/en_core_web_md-2.3.1.tar.gz#egg=en_core_web_md",
-    "spacy>=2.3,<2.4"
   ],
   "short": "Local Chatbot run on machine learning",
+  "end_user_data_statement": "This cog only stores anonymous conversations data; no End User Data is stored.",
   "tags": [
     "chat",
     "chatbot",

chatter/requirements.txt

@@ -1,12 +0,0 @@
-git+git://github.com/gunthercox/chatterbot-corpus@master#egg=chatterbot_corpus
-mathparse>=0.1,<0.2
-nltk>=3.2,<4.0
-pint>=0.8.1
-python-dateutil>=2.8,<2.9
-pyyaml>=5.3,<5.4
-sqlalchemy>=1.3,<1.4
-pytz
-spacy>=2.3,<2.4
-https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz#egg=en_core_web_sm
-https://github.com/explosion/spacy-models/releases/download/en_core_web_md-2.3.1/en_core_web_md-2.3.1.tar.gz#egg=en_core_web_md
-# https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-2.3.1/en_core_web_lg-2.3.1.tar.gz#egg=en_core_web_lg

chatter/storage_adapters.py

@@ -0,0 +1,71 @@
+from chatterbot.storage import StorageAdapter, SQLStorageAdapter
+
+
+class MyDumbSQLStorageAdapter(SQLStorageAdapter):
+    def __init__(self, **kwargs):
+        super(SQLStorageAdapter, self).__init__(**kwargs)
+
+        from sqlalchemy import create_engine, inspect
+        from sqlalchemy.orm import sessionmaker
+
+        self.database_uri = kwargs.get("database_uri", False)
+
+        # None results in a sqlite in-memory database as the default
+        if self.database_uri is None:
+            self.database_uri = "sqlite://"
+
+        # Create a file database if the database is not a connection string
+        if not self.database_uri:
+            self.database_uri = "sqlite:///db.sqlite3"
+
+        self.engine = create_engine(self.database_uri, connect_args={"check_same_thread": False})
+
+        if self.database_uri.startswith("sqlite://"):
+            from sqlalchemy.engine import Engine
+            from sqlalchemy import event
+
+            @event.listens_for(Engine, "connect")
+            def set_sqlite_pragma(dbapi_connection, connection_record):
+                dbapi_connection.execute("PRAGMA journal_mode=WAL")
+                dbapi_connection.execute("PRAGMA synchronous=NORMAL")
+
+        if not inspect(self.engine).has_table("Statement"):
+            self.create_database()
+
+        self.Session = sessionmaker(bind=self.engine, expire_on_commit=True)
+
+
+class AsyncSQLStorageAdapter(SQLStorageAdapter):
+    def __init__(self, **kwargs):
+        super(SQLStorageAdapter, self).__init__(**kwargs)
+
+        self.database_uri = kwargs.get("database_uri", False)
+
+        # None results in a sqlite in-memory database as the default
+        if self.database_uri is None:
+            self.database_uri = "sqlite://"
+
+        # Create a file database if the database is not a connection string
+        if not self.database_uri:
+            self.database_uri = "sqlite:///db.sqlite3"
+
+    async def initialize(self):
+        # from sqlalchemy import create_engine
+        from aiomysql.sa import create_engine
+        from sqlalchemy.orm import sessionmaker
+
+        self.engine = await create_engine(self.database_uri, convert_unicode=True)
+
+        if self.database_uri.startswith("sqlite://"):
+            from sqlalchemy.engine import Engine
+            from sqlalchemy import event
+
+            @event.listens_for(Engine, "connect")
+            def set_sqlite_pragma(dbapi_connection, connection_record):
+                dbapi_connection.execute("PRAGMA journal_mode=WAL")
+                dbapi_connection.execute("PRAGMA synchronous=NORMAL")
+
+        if not self.engine.dialect.has_table(self.engine, "Statement"):
+            self.create_database()
+
+        self.Session = sessionmaker(bind=self.engine, expire_on_commit=True)

chatter/trainers.py

@@ -0,0 +1,351 @@
+import asyncio
+import csv
+import html
+import logging
+import os
+import pathlib
+import time
+from functools import partial
+
+from chatterbot import utils
+from chatterbot.conversation import Statement
+from chatterbot.tagging import PosLemmaTagger
+from chatterbot.trainers import Trainer
+from redbot.core.bot import Red
+from dateutil import parser as date_parser
+from redbot.core.utils import AsyncIter
+
+log = logging.getLogger("red.fox_v3.chatter.trainers")
+
+
+class KaggleTrainer(Trainer):
+    def __init__(self, chatbot, datapath: pathlib.Path, **kwargs):
+        super().__init__(chatbot, **kwargs)
+
+        self.data_directory = datapath / kwargs.get("downloadpath", "kaggle_download")
+
+        self.kaggle_dataset = kwargs.get(
+            "kaggle_dataset",
+            "Cornell-University/movie-dialog-corpus",
+        )
+
+        # Create the data directory if it does not already exist
+        if not os.path.exists(self.data_directory):
+            os.makedirs(self.data_directory)
+
+    def is_downloaded(self, file_path):
+        """
+        Check if the data file is already downloaded.
+        """
+        if os.path.exists(file_path):
+            self.chatbot.logger.info("File is already downloaded")
+            return True
+
+        return False
+
+    async def download(self, dataset):
+        import kaggle  # This triggers the API token check
+
+        future = await asyncio.get_event_loop().run_in_executor(
+            None,
+            partial(
+                kaggle.api.dataset_download_files,
+                dataset=dataset,
+                path=self.data_directory,
+                quiet=False,
+                unzip=True,
+            ),
+        )
+
+    def train(self, *args, **kwargs):
+        log.error("See asynctrain instead")
+
+    def asynctrain(self, *args, **kwargs):
+        raise self.TrainerInitializationException()
+
+
+class SouthParkTrainer(KaggleTrainer):
+    def __init__(self, chatbot, datapath: pathlib.Path, **kwargs):
+        super().__init__(
+            chatbot,
+            datapath,
+            downloadpath="ubuntu_data_v2",
+            kaggle_dataset="tovarischsukhov/southparklines",
+            **kwargs,
+        )
+
+
+class MovieTrainer(KaggleTrainer):
+    def __init__(self, chatbot, datapath: pathlib.Path, **kwargs):
+        super().__init__(
+            chatbot,
+            datapath,
+            downloadpath="kaggle_movies",
+            kaggle_dataset="Cornell-University/movie-dialog-corpus",
+            **kwargs,
+        )
+
+    async def run_movie_training(self):
+        dialogue_file = "movie_lines.tsv"
+        conversation_file = "movie_conversations.tsv"
+        log.info(f"Beginning dialogue training on {dialogue_file}")
+        start_time = time.time()
+
+        tagger = PosLemmaTagger(language=self.chatbot.storage.tagger.language)
+
+        # [lineID, characterID, movieID, character name, text of utterance]
+        # File parsing from https://www.kaggle.com/mushaya/conversation-chatbot
+
+        with open(self.data_directory / conversation_file, "r", encoding="utf-8-sig") as conv_tsv:
+            conv_lines = conv_tsv.readlines()
+        with open(self.data_directory / dialogue_file, "r", encoding="utf-8-sig") as lines_tsv:
+            dialog_lines = lines_tsv.readlines()
+
+        # trans_dict = str.maketrans({"<u>": "__", "</u>": "__", '""': '"'})
+
+        lines_dict = {}
+        for line in dialog_lines:
+            _line = line[:-1].strip('"').split("\t")
+            if len(_line) >= 5:  # Only good lines
+                lines_dict[_line[0]] = (
+                    html.unescape(("".join(_line[4:])).strip())
+                    .replace("<u>", "__")
+                    .replace("</u>", "__")
+                    .replace('""', '"')
+                )
+            else:
+                log.debug(f"Bad line {_line}")
+
+        # collecting line ids for each conversation
+        conv = []
+        for line in conv_lines[:-1]:
+            _line = line[:-1].split("\t")[-1][1:-1].replace("'", "").replace(" ", ",")
+            conv.append(_line.split(","))
+
+        # conversations = csv.reader(conv_tsv, delimiter="\t")
+        #
+        # reader = csv.reader(lines_tsv, delimiter="\t")
+        #
+        #
+        #
+        # lines_dict = {}
+        # for row in reader:
+        #     try:
+        #         lines_dict[row[0].strip('"')] = row[4]
+        #     except:
+        #         log.exception(f"Bad line: {row}")
+        #         pass
+        #     else:
+        #         # log.info(f"Good line: {row}")
+        #         pass
+        #
+        # # lines_dict = {row[0].strip('"'): row[4] for row in reader_list}
+
+        statements_from_file = []
+        save_every = 300
+        count = 0
+
+        # [characterID of first, characterID of second, movieID, list of utterances]
+        async for lines in AsyncIter(conv):
+            previous_statement_text = None
+            previous_statement_search_text = ""
+
+            for line in lines:
+                text = lines_dict[line]
+                statement = Statement(
+                    text=text,
+                    in_response_to=previous_statement_text,
+                    conversation="training",
+                )
+
+                for preprocessor in self.chatbot.preprocessors:
+                    statement = preprocessor(statement)
+
+                statement.search_text = tagger.get_text_index_string(statement.text)
+                statement.search_in_response_to = previous_statement_search_text
+
+                previous_statement_text = statement.text
+                previous_statement_search_text = statement.search_text
+
+                statements_from_file.append(statement)
+
+            count += 1
+            if count >= save_every:
+                if statements_from_file:
+                    self.chatbot.storage.create_many(statements_from_file)
+                    statements_from_file = []
+                count = 0
+
+        if statements_from_file:
+            self.chatbot.storage.create_many(statements_from_file)
+
+        log.info(f"Training took {time.time() - start_time} seconds.")
+
+    async def asynctrain(self, *args, **kwargs):
+        extracted_lines = self.data_directory / "movie_lines.tsv"
+        extracted_lines: pathlib.Path
+
+        # Download and extract the Ubuntu dialog corpus if needed
+        if not extracted_lines.exists():
+            await self.download(self.kaggle_dataset)
+        else:
+            log.info("Movie dialog already downloaded")
+        if not extracted_lines.exists():
+            raise FileNotFoundError(f"{extracted_lines}")
+
+        await self.run_movie_training()
+
+        return True
+
+        # train_dialogue = kwargs.get("train_dialogue", True)
+        # train_196_dialogue = kwargs.get("train_196", False)
+        # train_301_dialogue = kwargs.get("train_301", False)
+        #
+        # if train_dialogue:
+        #     await self.run_dialogue_training(extracted_dir, "dialogueText.csv")
+        #
+        # if train_196_dialogue:
+        #     await self.run_dialogue_training(extracted_dir, "dialogueText_196.csv")
+        #
+        # if train_301_dialogue:
+        #     await self.run_dialogue_training(extracted_dir, "dialogueText_301.csv")
+
+
+class UbuntuCorpusTrainer2(KaggleTrainer):
+    def __init__(self, chatbot, datapath: pathlib.Path, **kwargs):
+        super().__init__(
+            chatbot,
+            datapath,
+            downloadpath="kaggle_ubuntu",
+            kaggle_dataset="rtatman/ubuntu-dialogue-corpus",
+            **kwargs,
+        )
+
+    async def asynctrain(self, *args, **kwargs):
+        extracted_dir = self.data_directory / "Ubuntu-dialogue-corpus"
+
+        # Download and extract the Ubuntu dialog corpus if needed
+        if not extracted_dir.exists():
+            await self.download(self.kaggle_dataset)
+        else:
+            log.info("Ubuntu dialogue already downloaded")
+        if not extracted_dir.exists():
+            raise FileNotFoundError("Did not extract in the expected way")
+
+        train_dialogue = kwargs.get("train_dialogue", True)
+        train_196_dialogue = kwargs.get("train_196", False)
+        train_301_dialogue = kwargs.get("train_301", False)
+
+        if train_dialogue:
+            await self.run_dialogue_training(extracted_dir, "dialogueText.csv")
+
+        if train_196_dialogue:
+            await self.run_dialogue_training(extracted_dir, "dialogueText_196.csv")
+
+        if train_301_dialogue:
+            await self.run_dialogue_training(extracted_dir, "dialogueText_301.csv")
+
+        return True
+
+    async def run_dialogue_training(self, extracted_dir, dialogue_file):
+        log.info(f"Beginning dialogue training on {dialogue_file}")
+        start_time = time.time()
+
+        tagger = PosLemmaTagger(language=self.chatbot.storage.tagger.language)
+
+        with open(extracted_dir / dialogue_file, "r", encoding="utf-8") as dg:
+            reader = csv.DictReader(dg)
+
+            next(reader)  # Skip the header
+
+            last_dialogue_id = None
+            previous_statement_text = None
+            previous_statement_search_text = ""
+            statements_from_file = []
+
+            save_every = 50
+            count = 0
+
+            async for row in AsyncIter(reader):
+                dialogue_id = row["dialogueID"]
+                if dialogue_id != last_dialogue_id:
+                    previous_statement_text = None
+                    previous_statement_search_text = ""
+                    last_dialogue_id = dialogue_id
+                    count += 1
+                    if count >= save_every:
+                        if statements_from_file:
+                            self.chatbot.storage.create_many(statements_from_file)
+                            statements_from_file = []
+                        count = 0
+
+                if len(row) > 0:
+                    statement = Statement(
+                        text=row["text"],
+                        in_response_to=previous_statement_text,
+                        conversation="training",
+                        # created_at=date_parser.parse(row["date"]),
+                        persona=row["from"],
+                    )
+
+                    for preprocessor in self.chatbot.preprocessors:
+                        statement = preprocessor(statement)
+
+                    statement.search_text = tagger.get_text_index_string(statement.text)
+                    statement.search_in_response_to = previous_statement_search_text
+
+                    previous_statement_text = statement.text
+                    previous_statement_search_text = statement.search_text
+
+                    statements_from_file.append(statement)
+
+            if statements_from_file:
+                self.chatbot.storage.create_many(statements_from_file)
+
+        log.info(f"Training took {time.time() - start_time} seconds.")
+
+
+class TwitterCorpusTrainer(Trainer):
+    pass
+    # def train(self, *args, **kwargs):
+    #     """
+    #     Train the chat bot based on the provided list of
+    #     statements that represents a single conversation.
+    #     """
+    #     import twint
+    #
+    #     c = twint.Config()
+    #     c.__dict__.update(kwargs)
+    #     twint.run.Search(c)
+    #
+    #
+    #     previous_statement_text = None
+    #     previous_statement_search_text = ''
+    #
+    #     statements_to_create = []
+    #
+    #     for conversation_count, text in enumerate(conversation):
+    #         if self.show_training_progress:
+    #             utils.print_progress_bar(
+    #                 'List Trainer',
+    #                 conversation_count + 1, len(conversation)
+    #             )
+    #
+    #         statement_search_text = self.chatbot.storage.tagger.get_text_index_string(text)
+    #
+    #         statement = self.get_preprocessed_statement(
+    #             Statement(
+    #                 text=text,
+    #                 search_text=statement_search_text,
+    #                 in_response_to=previous_statement_text,
+    #                 search_in_response_to=previous_statement_search_text,
+    #                 conversation='training'
+    #             )
+    #         )
+    #
+    #         previous_statement_text = statement.text
+    #         previous_statement_search_text = statement_search_text
+    #
+    #         statements_to_create.append(statement)
+    #
+    #     self.chatbot.storage.create_many(statements_to_create)

coglint/coglint.py

@@ -28,6 +28,10 @@ class CogLint(Cog):
         self.config.register_global(**default_global)
         self.config.register_guild(**default_guild)
 
+    async def red_delete_data_for_user(self, **kwargs):
+        """Nothing to delete"""
+        return
+
     @commands.command()
     async def autolint(self, ctx: commands.Context):
         """Toggles automatically linting code"""
@@ -35,7 +39,7 @@ class CogLint(Cog):
 
         self.do_lint = not curr
         await self.config.lint.set(not curr)
-        await ctx.send("Autolinting is now set to {}".format(not curr))
+        await ctx.maybe_send_embed("Autolinting is now set to {}".format(not curr))
 
     @commands.command()
     async def lint(self, ctx: commands.Context, *, code):
@@ -44,7 +48,7 @@ class CogLint(Cog):
         Toggle autolinting with `[p]autolint`
         """
         await self.lint_message(ctx.message)
-        await ctx.send("Hello World")
+        await ctx.maybe_send_embed("Hello World")
 
     async def lint_code(self, code):
         self.counter += 1
@@ -54,11 +58,7 @@ class CogLint(Cog):
 
         future = await self.bot.loop.run_in_executor(None, lint.py_run, path, "return_std=True")
 
-        if future:
+        (pylint_stdout, pylint_stderr) = future or (None, None)
-            (pylint_stdout, pylint_stderr) = future
-        else:
-            (pylint_stdout, pylint_stderr) = None, None
-
         # print(pylint_stderr)
         # print(pylint_stdout)
 

coglint/info.json

@@ -2,16 +2,15 @@
   "author": [
     "Bobloy"
   ],
-  "bot_version": [
+  "min_bot_version": "3.3.0",
-    3,
-    0,
-    0
-  ],
   "description": "Lint python code posted in chat",
   "hidden": true,
   "install_msg": "Thank you for installing CogLint! Get started with `[p]load coglint` and `[p]help CogLint`",
-  "requirements": ["pylint"],
+  "requirements": [
+    "pylint"
+  ],
   "short": "Python cog linter",
+  "end_user_data_statement": "This cog does not store any End User Data",
   "tags": [
     "bobloy",
     "utils",

conquest/__init__.py

@@ -0,0 +1,15 @@
+from redbot.core import data_manager
+
+from .conquest import Conquest
+from .mapmaker import MapMaker
+
+
+async def setup(bot):
+    cog = Conquest(bot)
+    data_manager.bundled_data_path(cog)
+    await cog.load_data()
+
+    bot.add_cog(cog)
+
+    cog2 = MapMaker(bot)
+    bot.add_cog(cog2)

conquest/conquest.py

@@ -0,0 +1,422 @@
+import asyncio
+import json
+import logging
+import os
+import pathlib
+from abc import ABC
+from shutil import copyfile
+from typing import Optional
+
+import discord
+from PIL import Image, ImageChops, ImageColor, ImageOps
+from discord.ext.commands import Greedy
+from redbot.core import Config, commands
+from redbot.core.bot import Red
+from redbot.core.data_manager import bundled_data_path, cog_data_path
+
+log = logging.getLogger("red.fox_v3.conquest")
+
+
+class Conquest(commands.Cog):
+    """
+    Cog for
+    """
+
+    default_zoom_json = {"enabled": False, "x": -1, "y": -1, "zoom": 1.0}
+
+    def __init__(self, bot: Red):
+        super().__init__()
+        self.bot = bot
+        self.config = Config.get_conf(
+            self, identifier=67111110113117101115116, force_registration=True
+        )
+
+        default_guild = {}
+        default_global = {"current_map": None}
+        self.config.register_guild(**default_guild)
+        self.config.register_global(**default_global)
+
+        self.data_path: pathlib.Path = cog_data_path(self)
+        self.asset_path: Optional[pathlib.Path] = None
+
+        self.current_map = None
+        self.map_data = None
+        self.ext = None
+        self.ext_format = None
+
+    async def red_delete_data_for_user(self, **kwargs):
+        """Nothing to delete"""
+        return
+
+    async def load_data(self):
+        """
+        Initial loading of data from bundled_data_path and config
+        """
+        self.asset_path = bundled_data_path(self) / "assets"
+        self.current_map = await self.config.current_map()
+
+        if self.current_map:
+            if not await self.current_map_load():
+                await self.config.current_map.clear()
+
+    async def current_map_load(self):
+        map_data_path = self.asset_path / self.current_map / "data.json"
+        if not map_data_path.exists():
+            log.warning(f"{map_data_path} does not exist. Clearing current map")
+            return False
+
+        with map_data_path.open() as mapdata:
+            self.map_data: dict = json.load(mapdata)
+        self.ext = self.map_data["extension"]
+        self.ext_format = "JPEG" if self.ext.upper() == "JPG" else self.ext.upper()
+        return True
+
+    @commands.group()
+    async def conquest(self, ctx: commands.Context):
+        """
+        Base command for conquest cog. Start with `[p]conquest set map` to select a map.
+        """
+        if ctx.invoked_subcommand is None and self.current_map is not None:
+            await self._conquest_current(ctx)
+
+    @conquest.command(name="list")
+    async def _conquest_list(self, ctx: commands.Context):
+        """
+        List currently available maps
+        """
+        maps_json = self.asset_path / "maps.json"
+
+        with maps_json.open() as maps:
+            maps_json = json.load(maps)
+            map_list = "\n".join(maps_json["maps"])
+            await ctx.maybe_send_embed(f"Current maps:\n{map_list}")
+
+    @conquest.group(name="set")
+    async def conquest_set(self, ctx: commands.Context):
+        """Base command for admin actions like selecting a map"""
+        pass
+
+    @conquest_set.command(name="resetzoom")
+    async def _conquest_set_resetzoom(self, ctx: commands.Context):
+        """Resets the zoom level of the current map"""
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        zoom_json_path = self.data_path / self.current_map / "settings.json"
+        if not zoom_json_path.exists():
+            await ctx.maybe_send_embed(
+                f"No zoom data found for {self.current_map}, reset not needed"
+            )
+            return
+
+        with zoom_json_path.open("w+") as zoom_json:
+            json.dump({"enabled": False}, zoom_json)
+
+        await ctx.tick()
+
+    @conquest_set.command(name="zoom")
+    async def _conquest_set_zoom(self, ctx: commands.Context, x: int, y: int, zoom: float):
+        """
+        Set the zoom level and position of the current map
+
+        x: positive integer
+        y: positive integer
+        zoom: float greater than or equal to 1
+        """
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        if x < 0 or y < 0 or zoom < 1:
+            await ctx.send_help()
+            return
+
+        zoom_json_path = self.data_path / self.current_map / "settings.json"
+
+        zoom_data = self.default_zoom_json.copy()
+        zoom_data["enabled"] = True
+        zoom_data["x"] = x
+        zoom_data["y"] = y
+        zoom_data["zoom"] = zoom
+
+        with zoom_json_path.open("w+") as zoom_json:
+            json.dump(zoom_data, zoom_json)
+
+        await ctx.tick()
+
+    @conquest_set.command(name="zoomtest")
+    async def _conquest_set_zoomtest(self, ctx: commands.Context, x: int, y: int, zoom: float):
+        """
+        Test the zoom level and position of the current map
+
+        x: positive integer
+        y: positive integer
+        zoom: float greater than or equal to 1
+        """
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        if x < 0 or y < 0 or zoom < 1:
+            await ctx.send_help()
+            return
+
+        zoomed_path = await self._create_zoomed_map(
+            self.data_path / self.current_map / f"current.{self.ext}", x, y, zoom
+        )
+
+        await ctx.send(
+            file=discord.File(
+                fp=zoomed_path,
+                filename=f"current_zoomed.{self.ext}",
+            )
+        )
+
+    async def _create_zoomed_map(self, map_path, x, y, zoom, **kwargs):
+        current_map = Image.open(map_path)
+
+        w, h = current_map.size
+        zoom2 = zoom * 2
+        zoomed_map = current_map.crop((x - w / zoom2, y - h / zoom2, x + w / zoom2, y + h / zoom2))
+        # zoomed_map = zoomed_map.resize((w, h), Image.LANCZOS)
+        zoomed_map.save(self.data_path / self.current_map / f"zoomed.{self.ext}", self.ext_format)
+        return self.data_path / self.current_map / f"zoomed.{self.ext}"
+
+    @conquest_set.command(name="save")
+    async def _conquest_set_save(self, ctx: commands.Context, *, save_name):
+        """Save the current map to be loaded later"""
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        current_map_folder = self.data_path / self.current_map
+        current_map = current_map_folder / f"current.{self.ext}"
+
+        if not current_map_folder.exists() or not current_map.exists():
+            await ctx.maybe_send_embed("Current map doesn't exist! Try setting a new one")
+            return
+
+        copyfile(current_map, current_map_folder / f"{save_name}.{self.ext}")
+        await ctx.tick()
+
+    @conquest_set.command(name="load")
+    async def _conquest_set_load(self, ctx: commands.Context, *, save_name):
+        """Load a saved map to be the current map"""
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        current_map_folder = self.data_path / self.current_map
+        current_map = current_map_folder / f"current.{self.ext}"
+        saved_map = current_map_folder / f"{save_name}.{self.ext}"
+
+        if not current_map_folder.exists() or not saved_map.exists():
+            await ctx.maybe_send_embed(f"Saved map not found in the {self.current_map} folder")
+            return
+
+        copyfile(saved_map, current_map)
+        await ctx.tick()
+
+    @conquest_set.command(name="map")
+    async def _conquest_set_map(self, ctx: commands.Context, mapname: str, reset: bool = False):
+        """
+        Select a map from current available maps
+
+        To add more maps, see the guide (WIP)
+        """
+        map_dir = self.asset_path / mapname
+        if not map_dir.exists() or not map_dir.is_dir():
+            await ctx.maybe_send_embed(
+                f"Map `{mapname}` was not found in the {self.asset_path} directory"
+            )
+            return
+
+        self.current_map = mapname
+        await self.config.current_map.set(self.current_map)  # Save to config too
+
+        await self.current_map_load()
+
+        # map_data_path = self.asset_path / mapname / "data.json"
+        # with map_data_path.open() as mapdata:
+        #     self.map_data = json.load(mapdata)
+        #
+        # self.ext = self.map_data["extension"]
+
+        current_map_folder = self.data_path / self.current_map
+        current_map = current_map_folder / f"current.{self.ext}"
+
+        if not reset and current_map.exists():
+            await ctx.maybe_send_embed(
+                "This map is already in progress, resuming from last game\n"
+                "Use `[p]conquest set map [mapname] True` to start a new game"
+            )
+        else:
+            if not current_map_folder.exists():
+                os.makedirs(current_map_folder)
+            copyfile(self.asset_path / mapname / f"blank.{self.ext}", current_map)
+
+        await ctx.tick()
+
+    @conquest.command(name="current")
+    async def _conquest_current(self, ctx: commands.Context):
+        """
+        Send the current map.
+        """
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        current_img = self.data_path / self.current_map / f"current.{self.ext}"
+
+        await self._send_maybe_zoomed_map(ctx, current_img, f"current_map.{self.ext}")
+
+    async def _send_maybe_zoomed_map(self, ctx, map_path, filename):
+        zoom_data = {"enabled": False}
+
+        zoom_json_path = self.data_path / self.current_map / "settings.json"
+
+        if zoom_json_path.exists():
+            with zoom_json_path.open() as zoom_json:
+                zoom_data = json.load(zoom_json)
+
+        if zoom_data["enabled"]:
+            map_path = await self._create_zoomed_map(map_path, **zoom_data)
+
+        await ctx.send(file=discord.File(fp=map_path, filename=filename))
+
+    @conquest.command("blank")
+    async def _conquest_blank(self, ctx: commands.Context):
+        """
+        Print the blank version of the current map, for reference.
+        """
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        current_blank_img = self.asset_path / self.current_map / f"blank.{self.ext}"
+
+        await self._send_maybe_zoomed_map(ctx, current_blank_img, f"blank_map.{self.ext}")
+
+    @conquest.command("numbered")
+    async def _conquest_numbered(self, ctx: commands.Context):
+        """
+        Print the numbered version of the current map, for reference.
+        """
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        numbers_path = self.asset_path / self.current_map / f"numbers.{self.ext}"
+        if not numbers_path.exists():
+            await ctx.send(
+                file=discord.File(
+                    fp=self.asset_path / self.current_map / f"numbered.{self.ext}",
+                    filename=f"numbered.{self.ext}",
+                )
+            )
+            return
+
+        current_map = Image.open(self.data_path / self.current_map / f"current.{self.ext}")
+        numbers = Image.open(numbers_path).convert("L")
+
+        inverted_map = ImageOps.invert(current_map)
+
+        loop = asyncio.get_running_loop()
+        current_numbered_img = await loop.run_in_executor(
+            None, Image.composite, current_map, inverted_map, numbers
+        )
+
+        current_numbered_img.save(
+            self.data_path / self.current_map / f"current_numbered.{self.ext}", self.ext_format
+        )
+
+        await self._send_maybe_zoomed_map(
+            ctx,
+            self.data_path / self.current_map / f"current_numbered.{self.ext}",
+            f"current_numbered.{self.ext}",
+        )
+
+    @conquest.command(name="multitake")
+    async def _conquest_multitake(
+        self, ctx: commands.Context, start_region: int, end_region: int, color: str
+    ):
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        try:
+            color = ImageColor.getrgb(color)
+        except ValueError:
+            await ctx.maybe_send_embed(f"Invalid color {color}")
+            return
+
+        if end_region > self.map_data["region_max"] or start_region < 1:
+            await ctx.maybe_send_embed(
+                f"Max region number is {self.map_data['region_max']}, minimum is 1"
+            )
+            return
+        regions = [r for r in range(start_region, end_region + 1)]
+
+        await self._process_take_regions(color, ctx, regions)
+
+    async def _process_take_regions(self, color, ctx, regions):
+        current_img_path = self.data_path / self.current_map / f"current.{self.ext}"
+        im = Image.open(current_img_path)
+        async with ctx.typing():
+            out: Image.Image = await self._composite_regions(im, regions, color)
+            out.save(current_img_path, self.ext_format)
+            await self._send_maybe_zoomed_map(ctx, current_img_path, f"map.{self.ext}")
+
+    @conquest.command(name="take")
+    async def _conquest_take(self, ctx: commands.Context, regions: Greedy[int], *, color: str):
+        """
+        Claim a territory or list of territories for a specified color
+
+        :param regions: List of integer regions
+        :param color: Color to claim regions
+        """
+        if not regions:
+            await ctx.send_help()
+            return
+
+        if self.current_map is None:
+            await ctx.maybe_send_embed("No map is currently set. See `[p]conquest set map`")
+            return
+
+        try:
+            color = ImageColor.getrgb(color)
+        except ValueError:
+            await ctx.maybe_send_embed(f"Invalid color {color}")
+            return
+
+        for region in regions:
+            if region > self.map_data["region_max"] or region < 1:
+                await ctx.maybe_send_embed(
+                    f"Max region number is {self.map_data['region_max']}, minimum is 1"
+                )
+                return
+
+        await self._process_take_regions(color, ctx, regions)
+
+    async def _composite_regions(self, im, regions, color) -> Image.Image:
+        im2 = Image.new("RGB", im.size, color)
+
+        loop = asyncio.get_running_loop()
+
+        combined_mask = None
+        for region in regions:
+            mask = Image.open(
+                self.asset_path / self.current_map / "masks" / f"{region}.{self.ext}"
+            ).convert("L")
+            if combined_mask is None:
+                combined_mask = mask
+            else:
+                # combined_mask = ImageChops.logical_or(combined_mask, mask)
+                combined_mask = await loop.run_in_executor(
+                    None, ImageChops.multiply, combined_mask, mask
+                )
+
+        out = await loop.run_in_executor(None, Image.composite, im, im2, combined_mask)
+
+        return out

conquest/data/assets/HoI/data.json

@@ -0,0 +1,3 @@
+{
+  "region_max": 70
+}

conquest/data/assets/ck2/data.json

@@ -0,0 +1,3 @@
+{
+  "region_max": 70
+}

conquest/data/assets/maps.json

@@ -0,0 +1,7 @@
+{
+  "maps": [
+    "simple",
+	"ck2",
+	"HoI"
+  ]
+}

conquest/data/assets/simple/data.json

@@ -0,0 +1,4 @@
+{
+  "region_max": 70,
+  "extension": "jpg"
+}