Feedback
Hello, in a moment, I'll give a few thoughts I had when I was using this model. In general, it was a good model albeit with a few faults.
Firstly though, here's my setup: I use latest staging sillytavern on my android with roleplay alpaca advanced formatting and no system prompt. I used Lewdiculous' Q5_K_M Imatrix gguf quant and use koboldcpp on free colab as backend. I also use a peculiar character card system wherein I put example dialogue in desc:
And I put all info from card into author note at depth 1:
Here's what I found during minimal testing:
Prose is great: wording from the model's not too simplistic nor complicated in wording, I like that.
Inconsistent Formatting: Formatting is consistent across all responses, though sometimes model mixes up formatting.
Great understanding of semi-complicated author note: Model can grasp author note's info albeit there is a slight difficulty in grasping her character (For instance, for the character Victoria, model assumes her character to refer other people as 'humans' despite the fact she is supposed to be a former human forcibly converted into a furry so char's not supposed to use such terminology.)
Decent personality grasping: While personality grasping from the model is decent, I feel the model needs improvement as such. For instance while testing said character card, the model did not accentuate the psychological struggle more clearly in 'inner thoughts' against char's embedded brainwashing despite being emphasized in author note and first message. In later messages, char's inner thoughts are read like a scientist's observation rather than an 18 year old tsundere's mind in conflict.
Traces of gptism and weird wording: The classic 'shivers down your spine' and variations thereof, and wording such as 'tingles' looks strange.
A bit of struggle in pov: Usually model's responses are in 3rd pov, but sometimes model mixes it up with 1st pov.
Context Limit: As said by others, the model's context is limited to 8k, though it can be remedied by using 'alpha value' roping via oobabooga.
Slight incoherence: probably sampling (Min_P: 0.05, Smoothing: 0.08) but sometimes, responses by model can become incoherent context wise, sometimes confusing user's responses with model's own.
Good instruct following: It's a given.
Overall, very decent model. Just needs some improvement in generating responses and understanding responses.
Responses (Min_P: 0.05, Smoothing: 0.08):
Gptisms:
Context limit (This is 9k filled out of 10k):
Formatting errors:
Incoherence:
Response that reads like a scientific observation:
