Add chatter README.md
Switch to medium spaCy model by default Correct paths for requirements.txt
This commit is contained in:
parent
9bd4a957c9
commit
dc829a3d5b
135
chatter/README.md
Normal file
135
chatter/README.md
Normal file
@ -0,0 +1,135 @@
|
|||||||
|
# Chatter
|
||||||
|
|
||||||
|
Chatter is a tool designed to be a self-hosted chat cog.
|
||||||
|
|
||||||
|
It is based on the brilliant work over at [Chatterbot](https://github.com/gunthercox/ChatterBot) and [spaCy](https://github.com/explosion/spaCy)
|
||||||
|
|
||||||
|
|
||||||
|
## Warning
|
||||||
|
|
||||||
|
**Chatter is a CPU, RAM, and Disk intensive cog.**
|
||||||
|
|
||||||
|
Chatter by default uses spaCy's `en_core_web_md` training model, which is ~50 MB
|
||||||
|
|
||||||
|
Chatter can potential use spaCy's `en_core_web_lg` training model, which is ~800 MB
|
||||||
|
|
||||||
|
Chatter uses as sqlite database that can potentially take up a large amount os disk space,
|
||||||
|
depending on how much training Chatter has done.
|
||||||
|
|
||||||
|
The sqlite database can be safely deleted at any time. Deletion will only erase training data.
|
||||||
|
|
||||||
|
|
||||||
|
# Installation
|
||||||
|
The installation is currently very tricky, and only tested on a Windows Machine.
|
||||||
|
|
||||||
|
There are a number of reasons for this, but the main ones are as follows:
|
||||||
|
* Using a dev version of chatterbot
|
||||||
|
* Some chatterbot requirements conflict with Red's (as of 3.10)
|
||||||
|
* spaCy version is newer than chatterbot's requirements
|
||||||
|
* A symlink in spacy to map `en` to `en_core_web_sm` requires admin permissions on windows
|
||||||
|
* C++ Build tools are required on Windows for spaCy
|
||||||
|
* Pandoc is required for something on windows, but I can't remember what
|
||||||
|
|
||||||
|
## Windows Prerequisites
|
||||||
|
|
||||||
|
Install these on your windows machine before attempting the installation
|
||||||
|
|
||||||
|
[Visual Studio C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
|
||||||
|
|
||||||
|
[Pandoc - Universal Document Converter](https://pandoc.org/installing.html)
|
||||||
|
|
||||||
|
##Methods
|
||||||
|
### Windows - Manually
|
||||||
|
#### Step 1: Built-in Downloader
|
||||||
|
|
||||||
|
You need to get a copy of the requirements.txt provided with chatter, I recommend this method.
|
||||||
|
|
||||||
|
```
|
||||||
|
[p]repo add Fox https://github.com/bobloy/Fox-V3
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Step 2: Install Requirements
|
||||||
|
|
||||||
|
In a terminal running as an admin, navigate to the directory containing this repo.
|
||||||
|
|
||||||
|
I've used my install directory as an example.
|
||||||
|
|
||||||
|
```
|
||||||
|
cd C:\Users\Bobloy\AppData\Local\Red-DiscordBot\Red-DiscordBot\data\bobbot\cogs\RepoManager\repos\Fox\chatter
|
||||||
|
pip install -r requirements.txt
|
||||||
|
pip install --no-deps "chatterbot>=1.1"
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Step 3: Load Chatter
|
||||||
|
|
||||||
|
```
|
||||||
|
[p]cog install Fox chatter
|
||||||
|
[p]load chatter
|
||||||
|
```
|
||||||
|
|
||||||
|
### Linux - Manually
|
||||||
|
|
||||||
|
Linux installation has not currently been evaluated, but Ubuntu testing is planned.
|
||||||
|
|
||||||
|
# Configuration
|
||||||
|
|
||||||
|
Chatter works out the the box without any training by learning as it goes,
|
||||||
|
but will have very poor and repetitive responses at first.
|
||||||
|
|
||||||
|
Initial training is recommended to speed up its learning.
|
||||||
|
|
||||||
|
## Training Setup
|
||||||
|
|
||||||
|
### Minutes
|
||||||
|
```
|
||||||
|
[p]chatter minutes X
|
||||||
|
```
|
||||||
|
This command configures what Chatter considers the maximum amount of minutes
|
||||||
|
that can pass between statements before considering it a new conversation.
|
||||||
|
|
||||||
|
Servers with lots of activity should set this low, where servers with low activity
|
||||||
|
will want this number to be fairly high.
|
||||||
|
|
||||||
|
This is only used during training.
|
||||||
|
|
||||||
|
### Age
|
||||||
|
|
||||||
|
```
|
||||||
|
[p]chatter age X
|
||||||
|
```
|
||||||
|
This command configures the maximum number of days Chatter will look back when
|
||||||
|
gathering messages for training.
|
||||||
|
|
||||||
|
Setting this to be extremely high is not recommended due to the increased disk space required to store
|
||||||
|
the data. Additionally, higher numbers will increase the training time tremendously.
|
||||||
|
|
||||||
|
|
||||||
|
## Training
|
||||||
|
|
||||||
|
### Train English
|
||||||
|
|
||||||
|
```
|
||||||
|
[p]chatter trainenglish
|
||||||
|
```
|
||||||
|
|
||||||
|
This will train chatter on basic english greetings and conversations.
|
||||||
|
This is far from complete, but can act as a good base point for new installations.
|
||||||
|
|
||||||
|
### Train Channel
|
||||||
|
|
||||||
|
```
|
||||||
|
[p]chatter train #channel_name
|
||||||
|
```
|
||||||
|
This command trains Chatter on the specified channel based on the configured
|
||||||
|
settings. This can take a long time to process.
|
||||||
|
|
||||||
|
|
||||||
|
## Switching Algorithms
|
||||||
|
|
||||||
|
```
|
||||||
|
[p]chatter algorithm X
|
||||||
|
```
|
||||||
|
|
||||||
|
Chatter can be configured to use one of three different Similarity algorithms.
|
||||||
|
|
||||||
|
Changing this can help if the response speed is too slow, but can reduce the accuracy of results.
|
@ -12,12 +12,18 @@ from redbot.core.commands import Cog
|
|||||||
from redbot.core.data_manager import cog_data_path
|
from redbot.core.data_manager import cog_data_path
|
||||||
|
|
||||||
|
|
||||||
class ENG_LG:
|
class ENG_LG: # TODO: Add option to use this large model
|
||||||
ISO_639_1 = 'en_core_web_lg'
|
ISO_639_1 = 'en_core_web_lg'
|
||||||
ISO_639 = 'eng'
|
ISO_639 = 'eng'
|
||||||
ENGLISH_NAME = 'English'
|
ENGLISH_NAME = 'English'
|
||||||
|
|
||||||
|
|
||||||
|
class ENG_MD:
|
||||||
|
ISO_639_1 = 'en_core_web_md'
|
||||||
|
ISO_639 = 'eng'
|
||||||
|
ENGLISH_NAME = 'English'
|
||||||
|
|
||||||
|
|
||||||
class Chatter(Cog):
|
class Chatter(Cog):
|
||||||
"""
|
"""
|
||||||
This cog trains a chatbot that will talk like members of your Guild
|
This cog trains a chatbot that will talk like members of your Guild
|
||||||
@ -32,7 +38,7 @@ class Chatter(Cog):
|
|||||||
path: pathlib.Path = cog_data_path(self)
|
path: pathlib.Path = cog_data_path(self)
|
||||||
self.data_path = path / "database.sqlite3"
|
self.data_path = path / "database.sqlite3"
|
||||||
|
|
||||||
self.chatbot = self._create_chatbot(self.data_path, SpacySimilarity, 0.45, ENG_LG)
|
self.chatbot = self._create_chatbot(self.data_path, SpacySimilarity, 0.45, ENG_MD)
|
||||||
# self.chatbot.set_trainer(ListTrainer)
|
# self.chatbot.set_trainer(ListTrainer)
|
||||||
|
|
||||||
# self.trainer = ListTrainer(self.chatbot)
|
# self.trainer = ListTrainer(self.chatbot)
|
||||||
@ -151,7 +157,7 @@ class Chatter(Cog):
|
|||||||
await ctx.send_help()
|
await ctx.send_help()
|
||||||
return
|
return
|
||||||
|
|
||||||
self.chatbot = self._create_chatbot(self.data_path, algos[algo_number][0], algos[algo_number][1], ENG_LG)
|
self.chatbot = self._create_chatbot(self.data_path, algos[algo_number][0], algos[algo_number][1], ENG_MD)
|
||||||
|
|
||||||
await ctx.tick()
|
await ctx.tick()
|
||||||
|
|
||||||
@ -162,6 +168,10 @@ class Chatter(Cog):
|
|||||||
Active servers should set a lower number, while less active servers should have a higher number
|
Active servers should set a lower number, while less active servers should have a higher number
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
if minutes < 1:
|
||||||
|
await ctx.send_help()
|
||||||
|
return
|
||||||
|
|
||||||
await self.config.guild(ctx.guild).convo_length.set(minutes)
|
await self.config.guild(ctx.guild).convo_length.set(minutes)
|
||||||
|
|
||||||
await ctx.tick()
|
await ctx.tick()
|
||||||
@ -173,6 +183,10 @@ class Chatter(Cog):
|
|||||||
Will train on 1 day otherwise
|
Will train on 1 day otherwise
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
if days < 1:
|
||||||
|
await ctx.send_help()
|
||||||
|
return
|
||||||
|
|
||||||
await self.config.guild(ctx.guild).days.set(days)
|
await self.config.guild(ctx.guild).days.set(days)
|
||||||
await ctx.tick()
|
await ctx.tick()
|
||||||
|
|
||||||
|
@ -8,5 +8,5 @@ sqlalchemy>=1.3,<1.4
|
|||||||
pytz
|
pytz
|
||||||
spacy>=2.3,<2.4
|
spacy>=2.3,<2.4
|
||||||
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz#egg=en_core_web_sm
|
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz#egg=en_core_web_sm
|
||||||
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz#egg=en_core_web_md
|
https://github.com/explosion/spacy-models/releases/download/en_core_web_md-2.3.1/en_core_web_md-2.3.1.tar.gz#egg=en_core_web_md
|
||||||
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.3.1/en_core_web_sm-2.3.1.tar.gz#egg=en_core_web_lg
|
# https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-2.3.1/en_core_web_lg-2.3.1.tar.gz#egg=en_core_web_lg
|
Loading…
x
Reference in New Issue
Block a user