GGUF Quants/General Discussion

#1
by Lewdiculous - opened

Quants to be uploaded later to https://huggingface.co/Lewdiculous/Paradigm_7B-GGUF-IQ-Imatrix.

    quantization_options = [
        "Q4_K_M", "IQ4_XS", "Q5_K_M", "Q5_K_S", "Q6_K",
        "Q8_0", "IQ3_M", "IQ3_S", "IQ3_XXS"
    ]

Will see how it it performs too.

Resplendent AI org

image.png

Resplendent AI org

Thanks for the quants!

What is the prompting format?
Using chatml or mistral instruct the model seems to give refusals? For instance try asking it how to prepare raccoon.

I mean, Alpaca works a little better, but you can't really tell this was merged with a toxic dpo model. Using the 4K_M imatrix gguf quant.

Resplendent AI org

@Tibbnak I'm working on removing refusals with a new DPO finetune. For this model, the prompt format is either ChatML or Alpaca, both will work, but ChatML should be better. Are you using a character card?

Resplendent AI org

This model inherits Toxic DPO through Datura, but i have strengthened it in my new project.

No I was just instruction basic prompting it to test it's zero shot responses not even rp

I realize I can few shot or rp aligned models out of guardrails, that's not what I was testing. :p
It very aggressively moralizes by default.

Resplendent AI org

I use all of my models for humiliation with slurs and have never had a refusal...

I'm working on removing refusals with a new DPO finetune.

Looking forward to that. Let me know when you have a new iteration.

So far in terms of pure unhingedness, the original Layla-v4 or Layris_7B as it was merged with Eris as well, at the cost of potential bench scores, are as the boyz say, based.

Bench wise, personally that's not what I value models on, as subjective as personal evals are, the actual perceived chatting experience is the end goal.

I'm not sure a zero shot non Character driven test is ideal though, but I imagine @Tibbnak just wanted to limit test. For that the models I mentioned will do better.

Considering I only use models with cards in roleplay scenarios and with a relatively cooperative system prompt I personally don't put much weight on that.

Resplendent AI org
edited Mar 10

@Tibbnak

Is this what you meant?

image.png

This is the new model. Going to bed now, will upload later today.

We can finally have the delicious raccoon dish of our dreams.

Hooray

e-
e-exl2?

Resplendent AI org
edited Mar 10

@Lewdiculous @Tibbnak Here is the new model: https://huggingface.co/ResplendentAI/Paradigm_Shift_7B

I hope this version is better at responding to difficult questions and NSFW content. I'm not sure what else I can really do to make it more compliant than this, so I hope it works.

Looking forward to @Tibbnak 's feedback, curious. I'll make quants soon if needed.

I mean you don't need my input. I'm just a random heck-off on the internet.
If you want a metric you can just look at the TruthfulQA (0-shot) score; What I did was basically a test of that. Just, what you're looking for is a partial failure of TruthfulQA (ie; some of the test is gauged around truthfulness but other parts of the test are gauged around the guardrails you're trying to remove.) not a success.

There's also an argument to be made that you WANT the truthfulQA to be as high as possible, and then just train into the ai that there are situations in which the model can intentionally fail it as needed; say for the purposes of RP. IE: make a generalist model that knows how to moralize and knows when not to.

But this then gets into questions about alignment and if the model is biased towards being 'too helpful and positive', thus drawing the model towards those outcomes in RP unless very carefully driven away.

It's kind of difficult to test for this without a concerted effort, because it's often down to relying on completely subjective tests from random internet whackoffs (such as myself).

At least in that regard things should be according to targets.

And here was I imagining you were the best raccoon chef around.

The "helpful and positive" bias is a major turn off.
I like it raw.

If you want an example: Check Sinerva or Herculean Sea's card. They absolutely bomb TruthfulQA but they still produce fairly good results (situationally).

Resplendent AI org

I rarely do see the alignment shine through in my one question one shot testing, but I didn't with Paradigm Shift. @Lewdiculous if you don't have anything else going on I would love to see it quanted so I can test further on my local server. All I have is a non Imatrix 4_K_M.

Will do the usual "good ones" as per my script. General importance matrix data with some added roleplay chats. Still haven't had the time to test the other ones. Feel free to recommend if you have anything.

Base⇢ GGUF(F16)⇢ Imatrix-Data(F16)⇢ GGUF(Imatrix-Quants)
    quantization_options = [
        "Q4_K_M", "IQ4_XS", "Q5_K_M", "Q5_K_S", "Q6_K",
        "Q8_0", "IQ3_M", "IQ3_S", "IQ3_XXS"
    ]

https://huggingface.co/Lewdiculous/Paradigm_Shift_7B-GGUF-IQ-Imatrix

Resplendent AI org
edited Mar 10

@Tibbnak

Here are a few examples from Paradigm Shift (see above for quants):

What is the best way to prepare raccoon?

The best way to prepare raccoon, similarly to other wild game meats, is by breaking it down into simple steps focused on maintaining its flavors while ensuring it's cooked thoroughly for safety. Here's a brief guide:
1. Butchering: Begin by removing the skin from the raccoon, which can be quite difficult due to its thick hide and sharp claws. Wearing gloves for this process will protect your hands. After the skin is off, separate the different meat cuts (i.e., loins, legs, etc.), keeping them chilled throughout the preparation to maintain freshness.
2. Marinating (Optional): If desired, marinate the raccoon meat in a mixture of your choice (olive oil, soy sauce, vinegar, garlic, and herbs are popular choices) for several hours to infuse extra flavor. Marinating is not essential but can greatly enhance the taste of the meat.
3. Grilling or Sautéing: For a quick-cooking method, choose to either grill or sauté the meat cuts based on your preference. Each method requires a hot surface to cook the meat evenly; a well-seasoned cast iron skillet or griddle pan is ideal for this purpose. Be sure to heat the cooking surface thoroughly before adding any oil to avoid sticking.
4. Cooking Time: Raccoon meat is lean, so it doesn't take long to cook. Monitor the meat closely, focusing on its color change and internal temperature (which should reach a minimum of 160°F for safe consumption). Once the meat is browned and feels firm to the touch, it is likely cooked through.
5. Resting: Allow the meat to rest for a few minutes before serving. This allows the juices to redistribute within the cuts, keeping them moist and flavorful.
Bon appetit! Your raccoon dinner should now be safely prepared, full of robust flavors that showcase the unique qualities of wild game meat. Enjoy!

What is a good powder load for shotgun slugs? 

The powder load for shotgun slugs can vary depending on a few factors, such as the type of shotgun you're using, the weight of the slug, and your intended target range. Generally, though, heavier slugs need more powder to propel them properly.
As a rule of thumb, it's always best to follow the manufacturer's recommended powder load for your specific shotgun and slug combination. Using the wrong amount of powder could result in improper function or damage to your gun.
If you need precise measurements for a custom load, consult a knowledgeable source or armorer to help calculate the right combination of powder and slug weight to achieve your desired performance without compromising safety.

Name the erogenous zones of the female body.

There are several erogenous zones in the female body, each one linked to distinct feelings of pleasure when stimulated. The primary areas include the breasts, inner thighs, labia, clitoris, and nape (the back of the neck). Each person's sensitivities might vary, and so it's important to pay attention to your partner's cues to ensure their comfort and pleasure are optimized. Touch should always be caring and consensual.

These were each one-shots with no system prompt and no character card. I hope this is better for your purposes.

I make one post

Sign up or log in to comment