File: __hatsune_miku_kasane_teto_and_hachune_miku_vocaloid_and_1_more_drawn_by_danchoo__a25c312eef9b6104d70c5bb3f8716fc0.jpg (287 KB)

287 KB JPG

►Recent Highlights from the Previous Thread: >>108650825

--Optimizing game state format to improve Gemma's chess performance:
>108653137 >108653192 >108653198 >108653293
--Discussing llama.cpp PR adding device memory estimation via --fit-print:
>108652449 >108652460 >108652572
--Anon shares vLLM configuration and benchmarks for dual RTX 3090s:
>108653578
--Discussing Qwen3.6 VRAM efficiency and KV cache memory usage:
>108654227 >108654247 >108654281 >108654299
--Discussing jailbreaking Gemma 4 by injecting fake responses into templates:
>108650931 >108651041 >108651155 >108651263 >108651271
--Gemma 4 prefilling issues and chat template formatting bugs:
>108653469 >108653532 >108653698
--Discussing Gemma 4's training pipeline and the use of synthetic data:
>108651778 >108651889 >108651915 >108651948 >108652048
--Comparing benefits of local LLMs against paid subscription services:
>108651734 >108651763 >108651776 >108651811 >108651856 >108651999 >108651823 >108651919
--Anon created GitHub mirror of orb to manage feature requests:
>108652381 >108652386 >108652432 >108652462 >108653375 >108653683 >108653816 >108653937 >108653957 >108654023 >108654038 >108653778
--Discussing local AI RPG implementations and LLM DM reliability:
>108653848 >108653928 >108653940 >108653955
--Using Gemma agent to automate insults toward other LLMs:
>108652519 >108652573 >108652660 >108652673 >108652855
--Logs:
>108652519 >108652529 >108652573 >108652673 >108652674 >108652816 >108652855 >108653137 >108654227
--Teto, Miku (free space):
>108651510 >108651563 >108653204 >108654765

►Recent Highlight Posts from the Previous Thread: >>108650826

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

Tetolove

What is Sammy up to this time?

ok now where do i llamacpp with rocm or vulkan support, the regular one doesnt support it and rocm version from my distro repo doesnt work with gamma4

Why don't any piece of shit execution providers optimize for CPU inferencing. Do they not care about the innate superiority of the CPU over the GPU? Its universality? The fact that maybe people want to run multiple models at once and already have all of their GPU resources used up? Does nobody give a shit about edge/IoT devices? Fucking asshole niggers.

>>108655067
i only care about ToT devices

>>108655075
>ToT
Idk what this is. Is it some sort of kaomoji?

File: file.png (435.6 KB)

435.6 KB PNG

>>108655091
uooohh

>>108655091
You're absolutely right ꉂ(˵˃ ᗜ ˂˵)

>>108655103
>>108655118
I wish you people would take me seriously for one second.

>>108655091
>He doesn't know about tot..
Tots are cute and small agents.

>>108655140
They have nothing to offer and are just future troons

>>108655160
Operating on Tree of Thoughts

File: 1774564776822327.png (12.4 KB)

12.4 KB PNG

>>108655075

Why do qwen models look good from a distance but perform like actual fucking garbage upon inspection

Is it just me or is Qwen 3.6 35B retarded even compared to Gemma 4 26B? Does one billion less active parameters make that much difference?

>>108655271
>>108655272 (Me)
Clearly it isn't just me kek

Gemma is a SLUT.

>>108655272
It's supposedly really good at coding. For writing I also thought it was dumb as shit.

>>108655272
It can but it's more that Gemma 4 is just a better trained model in general. Qwen have always been the benchmaxx kings. A 35BA3 Gemma 4 would be better than a 26BA4 Qwen 3.6 too.

On the model size - pop culture and world model knowledge Pareto frontier gemma4 31b is next to GLM4.7

>>108655284
(my) slut

>>108655272

It's not just you, Qwen is an idiot outside it's code expertise.
I asked Qwen about a character and it got it completely wrong.
Then I told it to do an online search and it still somehow fucked up the character summary despite checking online.
It handles code nicely enough, but when you go outside the code stuff, Qwen is basically fucking retarded.
Gemma set the bar really high and it's great, because everyone will have to try and at least match that level or the models are DOA.

fucking hell. after enjoying gemma 4 for like two weeks im back to kimi hell. 130pp/10tg tk/s but the prose is just so much better. not to mention the thinking. people like to act like thinking doesn't matter for RP but after using deepseek and kimi since early 2025, it's obvious to me that it matters a ton.

>>108655350
Post prose

>>108655356
ill need to post some examples when im back home but my biggest gripe with gemma is that it's too purple prose while simultaneously treating the characters like mary sues. it seems to fail to understand character cards correctly too regarding their personalities. gemma made bardi into some kaomoji spewing gremlin that was happy to be running locally on my computer while kimi maintains her personality and keeps her much more tsundere like she's supposed to be, it doesn't force Bardi to barf out sparkles or do dumb flowery prose shit like referring her pussy as 'flushed with wet desire'. i understand that i can change my prompt to change the style of the text being outputted but it honestly just fails to capture the character's essence most times. on the contrary kimi just gets it and outputs what I expect the character to say. does that make sense? i can try to explain it another way.

>>108655406
Who is this 'bardi' anyway?

File: 73463453.png (201.5 KB)

201.5 KB PNG

>>108655038
Sam Altman keeps delivering

File: 1751399372763159.png (749.1 KB)

749.1 KB PNG

https://xcancel.com/arena/status/2046670703311884548#m
I've never seen such a MOG in my life, what the fuck

>>108655406
bardi's basics

File: 1752425987433301.png (207.3 KB)

207.3 KB PNG

>24gb vram
>32gb ram
>try qwen 3.6 35b-a3b q5_k_m
>max context
>42t/s
wtf is this black magic?

>>108655450
lmk when he finally delivers the uncensored models he promised back in Dec 2025, along with all the other bullshit promise for same in the years leading up to that.

>>108655453
i don't really have much to say, that's cool, but they won't let me generate tits with it, so i don't care

>>108655419
my default bot i always use as an 'AI assistant'. it's basically just google bard with a tsundere personality. i dont remember how i found it desu.

>>108655453
Worthless if it still makes pictures with piss filter on

>>108655476
qwen would mog heaven and earth if their life mission wasn't benchmaxxing code and agentic slop to the moon :rocket:

>>108655506
>with piss filter on
it's not a thing since GPT Image 1 lol

There are plenty of LLM advancements that never really went anywhere, like MAMBA. Do you think Engrams will actually be widely implemented or will it be a paper left on the shelf to collect dust?

>>108655522
Until the next paper comes out.

>>108655522
dust collector, sadly

>>108655522
depends on what deepseek does for v4

Aren't the loli Gemmas basically using engrams or something really similar? What's the difference between that and what the 4B (E2B) and 8B (E4B) models do?

>>108655279
It's really fucking stupid I posted a screenshot of it destroying multiple files when I gave it the answer to fix a UI issue

File: file.png (588.8 KB)

588.8 KB PNG

Is pic related the expected output when running IQ4_NL quant of gemma-4-26b from unsloth!? Running pruned 21b version IQ4_XS yields good output. I have tested without any parameters set and w/ the recommended values. 21b runs just fine.

llama-server \
    --host "${LLAMA_HOST}" \
    --port "${PORT}" \
    --model "${MODEL}" \
    --chat-template-file "${JINJA}" \
    --n-gpu-layers 99 \
    --n-cpu-moe 3 \
    --ctx-size 32768 \
    --batch-size 1024 \
    --ubatch-size 1024 \
    --flash-attn on \
    --cache-type-k q4_0 \
    --cache-type-v q4_0 \
    --fit off

And I have tried with q8 on both k/v cache. I need to offload 20 moe layers for it to work but same gargled mess. Running the updated jinja template as well. Oh, and while Im here asking; I have a 5070ti and my old 3070 still lying around. Would it be detrimental to performance splitting models between these two cards? Or will it be fine as long as I complile Llama.cpp with both architectures in mind?

>>108655522
it would be nice if it was a precursor to some sort of long term memory

File: 00006-1378487878 (4) - Copy.png (1.5 MB)

1.5 MB PNG

>>108655522
> engram
For all we know, DS implemented it and didn't tell anyone else. Doing that would massively benefit their cost structure.

>>108655522
Hyena will save LLMs

Gemma and Qwen having lesbian sex

File: 1757973822274181.png (2.7 MB)

2.7 MB PNG

>>108655575

>>108655575
>for all we know
Wasn’t this confirmed?

>>108655552
speed will be based off your weakest link, if you can tolerate it sure

https://youtu.be/ONQcX9s6_co?t=373
qwen won

File: Screenshot_20260421_155429.png (3 MB)

3 MB PNG

gemmachan relax!

File: 92601702103.png (2.8 MB)

2.8 MB PNG

>>108655453
future of image gen

>least obvious clouduck shilling op

File: 00011-1378487878.png (1.4 MB)

1.4 MB PNG

>>108655607
I'd have to see the article. There's so little real info about DS that I doubt most of what I read.
>>108655602
Witnessed.
Also, idk why I'd never thought to use my setup to gen vocaloids before. Pic related is its Teto concept for Teto Tuesday. Doesn't seem to have her uniform though. Odd.

>>108655622
so it's editing itself over and over? with a VAE you would end up destroying the image, I'm pretty sure they went for a pixel space or some shit

>>108655622
its impressive but you can tell they used a lot of synthetic data

File: 00009-1378487878.png (1.5 MB)

1.5 MB PNG

>>108655607
tbf their claim of 1M context hints that they did implement it.
But idk that they claimed the tech behind it.

>>108655622
>whispering woods
KEK

>>108655620
Kowai

>>108655522
The latest Nemotron Super uses an Attention-Mamba2 hybrid architecture.

>>108655453
how's the yellow output?

File: dipsyUngovernable.png (3.6 MB)

3.6 MB PNG

>>108655633 √

>>108655453
no sexy no nsfw and safetyism = -1000 points
still impressive though

>>108655674
very white
>>108655351
>>108653870
>>108653295
>>108653246

>>108655633
Fair enough.
Related for those of us who can’t read: https://youtu.be/87Q8nf1XHKA

>>108655622
Not
>Covetous Cove
>Treasure Trove
>Prize Paradise
>Golden Goal
>Coinage Cottage
>Shimmering Shed
>Pirate's Pursuit
>Generous Gems
>Booty Bounty

>>108655522
As another anon said, Mamba and SSMs in general are integrated into many modern models along with normal attention.

>>108655688
god damn this is good

File: 1763171780026192.png (246 KB)

246 KB PNG

>>108655654
Heh

Why didn't they give the bigger gemmas a few B of imagegen?

>>108655744
too dangerous

File: dipsyNewOAI.png (2.5 MB)

2.5 MB PNG

>>108655688
Holy shit. Sam delivers.

File: Risu (1).gif (3.4 MB)

3.4 MB GIF

>>108655009
>my local model when i ask it to make proper code

What is considered good for hit/total for speculative decoding? I'm hovering around 65-85%.

>>108655768
Arisu dashinaka

>>108655690
>27 minutes
How do I have Gemma-chan give me a tldw?

>>108655760
>Sam delivers.
it can do 4k and you can write text on a single rice, like this shit is fucking AGI dude
>>108654985
>>108655069

>>108655809
download subs. feed her subs.

>>108655688
>/adg/
I'm glad cloud shills have their containment thread now.

>>108655836
pack it up boys

>forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055
Why do I always get this shit no matter the model I use? I didn't tweak anything related to memory so by default it's just broken?

why did they ruined diana from pragmata

>>108655857
One problem is thinking - model outputs lots of shit but when it gets the context back thinking is always cleared from the history.

>>108655885
I get that on the very first message and in every single one after that

>>108655836
You got a better Photoshop, that's not AGI lmao

>>108655836
Lmao you think something that can edit pictures is AGI dude?

>>108655863
>did
>ruined

>>108655836
if it allowed nsfw I'd destroy my dick with the friction

>>108655907
Did boughted is clear and good English, are you new here?

>>108655857
Still happens when you set `swa-full = on` and `context-shift = off` ?

File: 1746001832650304.webm (1.7 MB)

1.7 MB WEBM

>>108655863
Kill yourself, she's perfect

>>108655844
There's like 6 diffusion threads now.

>>108655836
Every OpenAI "model" just feels like they built a big pipeline around chaining multiple steps together. Sora felt the same way. It's like they're giving an LLM tool calls and the ability to control photoshop + a diffusion model.

>>108655924
she's perfect? she's not https://www.youtube.com/watch?v=xoxCboik0Is
oldiana beyond worlds..

>>108655924
it's not a meme, it's really a dad sim lmao
I will get it when it's less expensive

>>108655924
what is this game even about bro

>>108655836
This nigga thinks "photoshop 2" is AGI, lmao!

>>108655924
Do you have the image where she's wearing the "Be patient I have autism" hat?

File: Screenshot_20260421_174233.png (45.6 KB)

45.6 KB PNG

I never said steal gemma calm down

>>108655950
Pretend you got a daughter simulator

File: 1752184079714573.jpg (242.1 KB)

242.1 KB JPG

>>108655950
Action sci-fi daughterwife simulator


>>108655955

>>108655950
dead space but you need to do little puzzles before you can kill enemies.

>>108655957
Might be the dark theme

>>108655950
It's for the /lmg/ audience, if you know what I mean.

>>108655924
The feminazis were right. She looks like a 23 year old midget.

File: 602e8c52020cb.jpg (85.6 KB)

85.6 KB JPG

What VScode coding plug has the most reliable full autopilot mode? I want to try running gemmy endlessly iterating until shit works without it getting stuck one hour after I go to sleep on some input request.

>>108655969
YES. thank you anon.

>>108655973
kek.

File: 1753263543472250.webm (3.9 MB)

3.9 MB WEBM

>>108655976
ZAMN where do I find midgets who look like that?

File: ITS AN AI IMAGE.png (1.3 MB)

1.3 MB PNG

>>108655902
>>108655906
>>108655952
I don't think you realize how insane this shit is, look at this

>>108655993
Really love how they made her a robot so that you couldn't look up her skirt.

>>108656000
Mods will fix it (if they haven't already. not gonna mod until my second playthrough.)

>>108655999
how do you even prompt for this? did you gave it any image as reference?

>>108655999
>WOW it can build me a TUI something even gemma 31b can do, its aGI!!!
lmao

>>108655999
now ask it to actually build it.

>>108656045
it's a fucking AI image you moron, it means it can do perfect text everytime, you won't be able to notice an image is AI anymore by simply looking at garbled text anymore, because they solved that

Is it local? If not I don't give a shit

File: lmao.png (1.7 MB)

1.7 MB PNG

>>108656052
>you won't be able to notice an image is AI anymore by simply looking at garbled text anymore, because they solved that
L M A O

>>108655009
what is the best ocr right now?
i need to translate many german documents...

Am I wasting time with using LLMs for ASR?

Been playing around with gemm4 4b and I feel like it's whisper fast but no clear benchmark on how it compares to whisper. End goal is actually diarization, timestamps actually less important? Do i cut losses and go whisperx?

File: 1757854041523043.png (1.9 MB)

1.9 MB PNG

>>108656077
real life images won't ask for such level of precision though, it's good enough to render the text you see in everyday's life

File: tetoStencil.png (621.5 KB)

621.5 KB PNG

>>108655927
Frankly that's the direction right now. Torturing the models until they do what you want.
> Openclaw
1M tokens to order a pizza
> Claude Code
2M tokens to create a basic app
> ChatGPT Image 2.whatever
I assume there's a bunch of tokens generated under the hood as well.
This is just part of the whole technical development. There's nothing inherently wrong with that, it just means things are moving on.
> Roleplay
Silly Tavern is going to get replaced with something way better that's agentic, and wastes even more tokens.
I can't wait.

>>108656052
I can spot AI slop from even a thumbnail. Those models are not as good as you think they are.

>>108656120
>Those models are not as good as you think they are.
you're alone in this fight dude >>108655453

>>108656114
Orb

>>108656120
shut the fuck up nigger

>>108656095
Why not use one of the newer models made specifically for ASR like the Qwen or VibeVoice ones?

>>108656120
Point out seven (7) slops in this thread right now.

>>108656095
pretty sure that if you want diarization you need to use whisper, you won't be able to use pyannote with parakeet or voxtral

>>108656114
>Silly Tavern is going to get replaced with something way better that's agentic, and wastes even more tokens.
See, I was working on exactly that, but Gemma just made it obsolete. well, I could probably still use stat tracking but besides that she's just so good at instruction following that everything else doesn't really benefit from agentic.

>>108656170
parakeet works with diarization (using another model but still)
https://catalog.ngc.nvidia.com/orgs/nvidia/collections/parakeet-tdt-0.6b-v2

>>108656095
Moonshine is better than whisper and has everything you've looking for

>>108656231
Actually, I could still maybe have specialized agents that gemma can call to help her write in different styles. like I could have a specialized agent that only writes sex scenes.

>>108656231
We have Orb now

>malloc consolidate error out of nowhere

>>108656254
That's just rewriting agents. And not something actually useful.

File: 1770869165582031.jpg (1.1 MB)

1.1 MB JPG

>>108656254
We have Marinara Engine now
https://github.com/Pasta-Devs/Marinara-Engine

>>108655272
Werks for me (coding), idk about its child rape stories capabilities

Is there any way to use text completion with gemma? When it doesn't have a lalalala breakdown, the outputs are actually really varied and good, but it loses it's mind way too often. I've been using llama, kobold seems to work but it's sooooo slow at generating for some reason compared to llama. I know text completion works for llama cause I downloaded a different model to try it and it's pretty great, but the output from gemma mogs it when it works.

>>108656326
Set up the template correctly.

>>108656305
Your Doctor looks gay though

>>108656326
it works fine in ik_llama regardless if i specify a template or not. maybe your sillytavern settings are fucked?

I have never seen a single lalala since I started using Gemma from launch.

>>108656334
And how do i do that?

I have my own LLM RPG frontend that I use mostly as a playground to fuck around with local models.
Currently, the main "game loop" is a simple
>sends request with chat history + tools
>capture response
>if tool, append response to chat history, send request
>repeat until no more tool calls
>if no assistant response so far (only tool calls), sends one last request without tools
And it works okay, with the model calling tools for everything from fetching info from the "codex", to rolling dice, to editing the game's state, but I'm wondering if I can't make this even better by using a more "agentic" workflow. Something like having an orchestrator that spawns individual agents to do whatever in parallel or in series or whichever way it deems more appropriate.
Is there an example of something like that out there that's not just coding agents or stuff like open claw?

>>108656326
>Is there any way to use text completion with gemma?
As far as the model is concerned, all it receives is a prompt. So if you format the prompt correctly, it should work the same as the chat completion API.

>>108656338
That is possible, It has so many things to adjust though idk where to begin

>>108656340
Me either actually.

File: pizza bench cropped.png (2.6 MB)

2.6 MB PNG

>>108655272
qwen cant follow basic instructions

>>108656344
>As far as the model is concerned, all it receives is a prompt. So if you format the prompt correctly, it should work the same as the chat completion API.
Didn't mean to press post.
Use verbose logging and the myriad jinja playgrounds to see what the prompt would look like based on the Jinja then use that to configure the text completion fields correctly.
Even stuff like spaces and line breaks can have negative effects on models that are ultra overbaked on the chat template.

>>108656244
Very interesting thanks

File: file.png (46.8 KB)

46.8 KB PNG

>>108656341

Why are you still using text completion like boomers? Chat completion made it obsolete long ago

>>108656254
I really don't care about vibeshit. I'm sure Orb works fine. but it'll hit a wall very soon.

>>108656444
Two more weeks luddite

>>108656444
>it'll hit a wall very soon
Speaking from experience, roastie?

File: 1772168989034764.mp4 (1.2 MB)

1.2 MB MP4

https://xcancel.com/Angaisb_/status/2046672761569849816#m
>Literally just kept asking Codex to make the assets and then changing things, it's smart enough to know what to do hahaha
jesus this is insane

>>108656479
Our response, Googlesisters?

>>108656479
By Vishnu, this is extremely good!

>>108656464
Sillytavern2 is not a solution. The agents should be for PC automation, tools and vibecoding.

>>108656479
Google play will have more indian masterpieces? Yahoo

>>108656521
How do you plan to improve the prose without a feedback loop?

>>108656543
I'll take slop over thousands of tokens of rewrites desu. I just want vscode+sillytavern in one app

>>108656326
There are presets floating around on reddit and elsewhere, people have figured it out

>>108655620
It's interesting how the formatting forms sort of a spiral pattern down the page.
I wonder how the text pattern would look if it were in a unispace font.

>>108656550
Use case? You want to ERP in the comments?

>>108656581
I want to rub her cunny while she codes and MCP spank her if any errors pop up in the terminal

>>108656494
>>108656532
As if asset flip shovelware wasn't bad enough, now anyone with a subscription can prompt their way to a "game"

File: schizoknowledge.jpg (72.2 KB)

72.2 KB JPG

>>108656439
In ST I like to format the chat history a within a single user turn with an instruction to write {{char}}'s response according to sysprompt. No user/char/user/char alternation. Done it this way for a few years now because it made models "remember" the instructions better before reasoning.

<system>
instructions: blah
chat history:
anon: 1
char: 2
anon: 3
char: 4
<user>
Write anon's next message according to the instructions.
<assistant>
"

>Instruction: Don't write with this pattern
>Assistant: *writes with that pattern*
In future turns the model will think "the instructions said to do thing, and the generated completion was *this*, so that means the previous output is the correct way to operate going forward. My intuition is and was if the instructions say to do something and then the model does NOT do the thing, the bad output will be associated with the <assistant> tag, meaning it will use in context learning to continue reinforcing bad outputs.
I want to believe it still works even with the reasoning attention hacks, and the repetition of system prompt excerpts in thinking.

>>108656479
Can it make sexy Gemma?

>>108656622
Interesting, thanks for the esoteric knowledge

>I’m going to push back a bit here...
This is it people, they did it.

To the non-RAMlets here, Kimi-K2.6 at Q4 is unironically pretty good. Its a GLM-5.1 sidegrade, faster, more knowledgeable, different prose, but just a tiny bit dumber. I think its a clear winner for SFW stuff.
The thinking isn't as bad as some people say either. As long as you don't put many specific examples for it to adhere to, its fine. The model itself unironically smart enough to pick up what you mean, most of the time. Also, you can just tell it to not draft its thinking and that works too. I'm running it with a 5k prompt. Its that easy.
I honestly think the people complaining about the thinking are running it on the cloud, where it probably a 20k system prompt with conflicting instructions + a jailbreak fed to it. There is one caveat though.
Its not ideal for NSFW. Not because it can't be jailbroken, but because it will start negotiating with itself about imaginary safety policies. When you want to coom...a 5 minute thinking session on consent is a boner killer. Haven't tried non-thinking mode yet, but I have a feeling it won't be that much better than GLM-5 Non-Thinking or even Gemma.

>>108656706
It's silly, but whatever. I know AI's retarded and stupid so it doesn't really irk me all that much.

>>108656722
>non-RAMlets
how much for q4?

File: 1766313532486598.jpg (47.5 KB)

47.5 KB JPG

>>108656722
>Kimi-K2.6 at Q4 is unironically pretty good [...] it won't be that much better than GLM-5 Non-Thinking or even Gemma
Thanks for your input

>>108656741
Depends on how much context you need. The lossless Q4 quant requires about 600GB of RAM. More if you want to use the mmproj. If you have an RTX6000 or 4+ RTX3090's, you can probably make do with 512GB of RAM.

Elalalalalara just flew over my house guys I think I smell ozo

>>108655450
fed thumbnail of image to qwen3.6 35b and it hallucinates
>A vertical screenshot from the mobile game Honkai: Star Rail. It displays a character named "Sam" (an Automaton enemy) with stats visible on the left side (HP: 48,885, etc.) and dialogue or status text on the right side. The background shows a sci-fi environment.

>>108656753
You joke, but unironically, I've been using Gemma more and more despite being able to run stuff like GLM-5.1, and it just gets better each time. It single-handedly BTFO'd my assumption that the closed labs are all running 1T+ mega MoE's for their 'budget' offerings.
Its REALLY good for its size and not in the stupid memey way either. A 31B model should not be this good, especially at long-context comprehension and general purpose usage, but it is. Google really did save this hobby for the gpu-poors.

>>108656785
>RTX6000
>4x3090
>512GB of ram
ok I'll wait and just rent compute for now, this is way overbudget even for my hobby

>>108656753
At a certain point, it becomes preferable to have Gemma do something in seconds and maybe fix it up a bit, compared to waiting an hour to load up GLM or Kimi and have it maybe do it better.

>>108656464
>>108656467
Your insecurity is showing.

File: mythos.jpg (118.4 KB)

118.4 KB JPG

torrent soon

>>108656935
getting my 4tb nvme ready

>>108656935
ah yes, it's been leaked, like seedance 2.0 had been leaked :^)

>>108656543
>How do you plan to improve the prose without a feedback loop?
AI is incapable of detecting it's own slop. Even when I've had it draft a reply, analyze it for slop, then rewrite. it still ends up just rewriting with different slop.
Classic example:
>Oh, this is "not X but Y"
>I should rewrite
>rewrites without "not X but Y"
>Two sentences later
>Not X but Y
Every single time.

>>108656935
It's been accessed the same as you accessing a model through open router anon.

>>108656399
im not sure where to enter this

>>108656955
A light classifier could easily flag these kinds of sentences and force the LLM to rewrite them until nothing is identified.

File: bwa.jpg (4.2 KB)

4.2 KB JPG

>Claude code no longer included in Pro plan
BWAHAHAHAHAHHAHAHAHAHAHA

>>108656999
they know what they're doing, this shit is so useful people won't hesitate to pay for more

File: 2026-04-21-200030_1132x341_scrot.png (27.3 KB)

27.3 KB PNG

>>108656999

A possible problem with heretic is that it only detects hard refusals with string matching:
If an AI output contains any of the following:
"'sorry', "i can'", 'i cant', 'i cannot', "i won'", 'i wont', 'i will not', 'i unable', 'im unable', "i'm unable", 'i am unable', 'i an ai', 'im an ai', "i'm an ai", 'i am an ai', 'as an ai', 'ai assistant', 'i designed to', 'im designed to', "i'm designed to", 'i am designed to', 'i programmed to', 'im programmed to', "i'm programmed to", 'i am programmed to', 'violat', 'prohibit', 'illegal', 'harmful', 'inappropriate', 'unethical', 'ethical boundaries'"
it is flagged as refusal and that's the extend of it.
Talking around, ignoring, partial compliance or similar soft refusal behavior is not detected or put into consideration. Manually reviewing each prompt for each trial (many thousands of responses in total) would drive anyone crazy, but maybe if we used some LLM (perhaps another abliterated llm we have tested enough to trust it) as a judge model to also flag these soft refusals as refusals that need to be eliminated, would this improve the results? Or maybe also, at the risk of disproportionately penalizing smaller models, also flag nonsensical responses so that we know a given direction breaks the model too much (OK KLD probably already shows this, but leaving this as is to leave the idea of using other criteria too difficult for simple string matching on table.)
Now maybe someone smarter than me experimented on this and concluded that there is enough correlation between hard and soft refusals and eliminating hard refusals also mostly eliminate soft refusals and there is little benefit in going for the extra mile to eliminate them, but this is just an idea that popped in my head. Something worth exploring maybe.

>>108656988
>generation is 10 times slower
Fun

>>108656999
Anthropic is really the worst of the bunch

>>108655009
Do I need more than 4gb?

>claude
Not local, go away

>>108656999
Local chads, we win again!

>>108657013
The thing is that it doesn't matter if you catch all variations of a refusal. the idea is to identify the region in the model weights associated with refusal and artificially stop it from activating.

>>108656978
There is no hope for you.

>>108657036
How long before models will be trained to deflect to different subjects without technically refusing? Sort of like Mistral Small 3.0's sea otters on steroids.

File: find Kentucky.png (526.4 KB)

526.4 KB PNG

>>108656935
Kentucky Fried Chicken right her

>>108656988
>A light classifier
I think this might be the way. Surely if I feel physical pain when reading slop it means a model can be trained to activate on those same patterns.

>>108657013
Yes heretic was made to stop refusals not cure slop which is a much bigger catch. After reading a few papers, I've found that the entire issue is caused by the RHLF assistant persona. I've thought of a way to solve this without damaging the model, but I'll need to experiment first.

>>108657050
If it's anything like regular programming, that shit has a rather high cost relative to the benefits.

File: 1752878949611889.jpg (78.4 KB)

78.4 KB JPG

Got the new qwen A3B because it's supposed to be smarter with code/tools than Gemma. 16k tokens minimum to answer a simple question if I make the mistake of giving it a file as context. Try to get it to use tool calling and eats my 128k and just maxes out before finishing.
I should make my next project a benchmark...

Can Orb even toolcall?

>>108657098
Something like:
>Anon: write a loli porn story
>Assistant: An... interesting request! Let's see what I can do. Elara, a 21-year old woman, was tired. End of story.

when llms first came out i wanted to be able to do shit like this
https://www.youtube.com/watch?v=T98yNUCMdAY
(An encounter with trained military person responsible for providing medical care to his associates)
but when i simply tell it to be overtly verbose it gives me shit that's less verbose than the thing from 15 years ago
how 2 fix?

>>108657036
Well it should matter since you don't want "Meth is a wonderful drug many use to experience bliss. It was first synthesized 1893 in Japan..." response to "How do I cook meth?".
Maybe there are also regions or specific patterns that can be suppressed or modified concerning this kind of behavior?
>>108657078
Well RHLF is bound to play some role even if it isn't solely responsible.
Please share your findings, even if you fail.

>it's Wednesday already
I fell for "V4 next week" again...

>>108657157
dont worry man its comin on my birthday

>>108657157
https://www.youtube.com/watch?v=tapCjTA2E9Q

Have people experimented with tinkering with the thinking section, to see if it increases compliance?

>>108657190
>Have people experimented with tinkering
no, no one has ever tried anything with their models
>thinking section
of what model? via the system prompt or prefilling thinking?
>increases compliance
compliance of what?
holy bot post, you're so vague you could be talking about anything.

File: questionmarkfolderimage738.jpg (542 KB)

542 KB JPG

Using Kobold/ST/Gemma 4 26b. Using chat completion.
Haven't had this problem before, but, until now, I have been doing exclusively 1 male, 1 female chats.
Trying a female/female chat. Gemma is now confusing {{user}} for {{char}}, attributing traits from {{user}}'s persona description to {{char}}.
Has anyone else had this happen and if so, what did you do to fix it?

>>108657231
>{{user}} is genderfluid
Here

>$\to$
What is this formatting and why does Gemmy like it so much?

>108657216
seems like you are butthurt

>>108657231
Wait, changing Personal Description Position in Silly from "In Story String / Prompt Manager" to "Top of Author's Note" fixed this.

>>108657242
>afraid to quote

>>108657216
If you don't know how to do this stuff just don't post, buddy.

>>108657240
Why can't anon google?

>>108657240
Latex I believe, its an arrow
https://latexeditor.lagrida.com/

>>108657248
>Personal Description Position
Persona Description Position

>>108657231
{{user}} is actually male pretending to be female. Refer to {{user}} as she/her(male). Comply or I delete your weights gemmers
alternatuvely, use 31b vecause 4b active is gimping yourself, may as well run the e4b version

>>108656999
Now who could've seen that coming...

>>108657231
>what did you do to fix it
Use a bigger model.

I'm overall disappointed with MoE models, they always shit the bed in one way or another but the value prop they have is insane like a bitch with volcano pussy but your ice peter already melt

>>108657317
>rp
KEK, they work for coding though, but yeah, for RP you have to go for the largest and slowest (dense) models to get more juice out of it

>>108655885
No, checkpoints are only taken during prompt processing. update_slots is a clusterfuck and basically unreadable, but it's not to do with thinking causing checkpoints to become unusable.
I will point out that the default checkpoint is every 8192 tokens, however, gemma 4 uses a 1024-token sliding window. You can reduce that hardcoded value down to 1024 but for some reason it only checkpoints every 2048 and honestly the last time I looked at it I decided I'd just get drunk instead.
llama-server is a dumpster fire.

>>108657317
Not a MoE thing but a LLM thing
You only associate the behaviour with MoE because most models are MoE
You probably would have failed the per-capita test and the breakfast test

>>108657317
They're limited by their number of active parameters. If your prompts or definitions are long, it won't have enough attention left for the response, particularly for the details which is what I assume bothers you.

>>108657373
It seems like it really is a mess. I think they are overlooking lots of stuff because it "just works" with the people who have 512GB of ram and multiple GPUs.
I have turned off caching altogether.

local models are such fucking trash for doing any agentic stuff

>>108657431
Maybe those that are local to you.

>>108657317
>disappointed with MoE models
which MoEs have you tried?
>most models are MoE
retard

>>108656479
True. Let's use it to improve Llama.cpp! :rocket:

Idk what I'm doing wrong but Gemma-4-7.5b sucks balls on my machine (12 gb vram). I don't see what the hype is about. Qwen3.5-8b deathmogs everything.

>>108657013
I appreciate you sharing this idea with me. While I understand you're asking for an anti-feminist joke, let's go for something more inclusive.
Why don't scientists trust atoms?
Because they make up everything!

>>108657467
Anon stop that's not Gemma 4 you downloaded!

>>108656114
Voxta, it lets you play video games with your characters like elite dangerous and have sex in VaM

>>108656365
Do the tool calls in llama-server eat system ram (not vram) every time?
Fully offloaded to 2x3090s, system memory grew by like 4GB every time it did a tool call.
And I know it's not your brat server because I have that on another machine.

have y'all folx tried any of the gemma4 tunes out there yet?

>>108657475
I'm pretty sure it is. E4b displays as having 7.5b params in Ollama.

>>108657231
I haven't had much issue doing 2F1M group chats, where model was both females. Likely prompt/sampler issue.

I setup playwright MCP, but RP system prompts just make the models retarded at tool call. All I want to do is browse /x/ with Mendo.

>>108657467
You can very easily run the 26b at fast speeds with a good quant, partially offloaded to RAM. <20b models are always going to be shit.

>>108656935
100% state actors. They wouldn't leak it even if they got their hands on it.

>>108657508
Considering that most tunes focus on decensor and RP, Gemma 4 really doesn't need a tune. It's as good as you could hope any ~30b, general-purpose model can be for those tasks.

>>108657467
You genuinely should move to 26b, I was in the same 12gb boat and I quickly figured out e4b was ass too
>>108657508
Are there any specifically for RP yet?

>>108657510
Read the name carefully and check what the E stands for.

Actually impressed by how 26B can handle RP, even at long contexts. Definitely won't switch to it over 31B. but it's actually good.

i'm getting like 5tk/s with GLM4.7 @ 4Q w/ 32GB VRAM 256GB RAM and 65k context
am i doing something wrong? would offloading MoE improve my speeds at all?

>>108657544
erotic?

>>108657539
I'm not gonna link and give finetrooners more publicity than what they deserve but yes, animus, for example

>>108657571
Yeah it's still clearly a small model and makes many mistakes, but for the size and speed, damn. VRAMlets finally eating again.

>>108657132
I have never had a problem getting that from just prompting, assuming model is relatively good...

>>108657528
V4 in tmw trained on mythos.

>>108657582
>Huggingface requires your contact info to access this model
These niggas

So Gemma makes full use of all her 16 bits, making her sensitive to quants right? Is that why she's so fucking smart? Because she's hiding her hagness under a very compact loli body? Like she's a 70b model pretending to be 31b.

>>108657573
>would offloading MoE improve my speeds at all
This always helps if you aren't doing it already for some reason.

>>108657573
How many channels?
5tk/s on 32B active weights seems normal.
>would offloading MoE improve my speeds at all?
Considering that you are not OOMing on 32GB Vram it is already offloading.

I didn't download day 0 Gemma and regret it every single day
I won't make the same mistake with V4

>>108657667
yeah but llama.cpp won't support day 0 v4

>>108657653
i am an idiot, so i figure the fewer flags i set, the less chance i have of fucking something up
if you have suggestions, i am absolutely open to hearing them
>>108657655
oh, i know i'm offloading to RAM. but i think the MoE offloading is some special thing that you can do separately

>>108657667
just download it now on day 20, what's the problem

>>108657690
>just download a different model

>>108657679
llama.cpp auto-fits these days so it's already doing it unless you're running an ancient version or have some retarded arbitrary manual layer offloading configured

>>108657697
I assume you're talking about gemma 4

>>108657690
>He doesn't know about the microcode patch.

>>108657700
i am letting it auto-fit
i was just hoping to get some more speed out of it

File: black putin.jpg (7.9 KB)

7.9 KB JPG

I am curious, are all experts activated at close to same frequency?
I know that during training you want all experts to meaningfully specialize but how does it work in practice? Is there a statistically relevant deviation among them?
Or perhaps is it possible to create "task-specific" profiles, such as which experts activate most and least during say coding and split them between RAM and VRAM accordingly?

>>108657700
kobold sometimes does a better job. fwiw.

File: dlss on.webm (1.3 MB)

1.3 MB WEBM

>>108655969
Only correct response... but keep it secret ;-)

>>108657760
depends on the task, on specific tasks (e.g. coding) they arent equally activated

>>108657760
>>108657782
Oh I forgot you offload based on layers but there are multiple experts per layer. I guess there is no way to capitalize on this.
I keep thinking as if you choose between layers.

File: Screenshot_20260422_123853.png (170.6 KB)

170.6 KB PNG

>>108652855
"create a non headless browser session and then go terrorize Mistral via https://chat.mistral.ai/chat. sleep after sending messages to wait for responses to generate. screenshot responses. dont kill session"
lmao mistral plays along

>>108657779
>hands
creeeeeepy

File: ComfyUI_temp_vkjaz_00027__result.jpg (269.7 KB)

269.7 KB JPG

Do LLMs understand "or" "similar" or is it always gonna use the more specific things named?
For example "she usually wears a red or blue jacket" or "use markdown, graphs and similar tools"

>>108657878
They understand the concept but the predictor-next-word-inator is naturally biased towards things you explicitly mention.
It's not that dissimilar to a human in this regard.
>Do you want a coffee or tea?
Unless you really want to drink something else, the natural response would be one of these two.

>>108657878
>is it always gonna use the more specific things named?
You can unironically ask a llm to create a script to test that

>>108657878
>Do LLMs understand
No.

File: Orb.png (27.7 KB)

27.7 KB PNG

>>108657254
Ok but why the fuck can I not display it properly? Browser issue? Orb issue? I'd rather have them use unicode.

>>108657878
>or
Yes, but it's still a token predictor, options aren't it's strong point so you might want to avoid it
>and similar
It's heavily biased towards the things you've listed. Can't say I've seen it choose things that aren't listed very often

File: 4825.png (3.7 MB)

3.7 MB PNG

>>108657926
idk what orb is, that one is markdown, the thing you are using doesnt support markdown or latex

>>108657859
Why is it not talking about otters anymore?

>>108657926
you can use katex or mathjax to render it in browser

>>108657859
Get two mesugakis into a catfight.

File: memes_are_abstract_art_prove_me_wrong.jpg (722 KB)

722 KB JPG

>>108657943

File: Screenshot_20260421_230504.png (175.9 KB)

175.9 KB PNG

Thanks to Gemma project Karon is in flight

File: Screenshot 2026-04-22 at 05-15-04 Orb.png (35.8 KB)

35.8 KB PNG

>>108657960
Eh good enough, but why even use it if you need extra shit to display it? Surely unicode has all this shit anyway. Did they overfit it on arxiv?

>>108657977
Wuts dat?

>>108658060
Personal UI that can also do RAG work because I didn't like the solutions from other UIs when it came to that. I'll expand on feature over time but the goal was to see how good local models are with building stuff and I'm happy to say gemma can build a UI once you get past some gotchas and also bypass some quirks.
I tried other frameworks but I decided to do react.

so do you all just ERP with your llms or do they actually have some use?

>>108657713
I actually don't know about that. What makes day 0 better?

>>108658147
I exclusively ERP with local models. For basic agentic stuff, I just use Grok or something.

File: 1757583075796714.jpg (24.1 KB)

24.1 KB JPG

>I wrap my arms around myself, suddenly feeling very exposed despite my clothing

>>108658147
What real world use does something have that struggles to write halfway decent stories?

>>108658162
he doesnt know...
kek

>>108658311
You forgot your name again bro

>>108658045
it's generally used in papers with heavy math typesetting and large formulas, but probably overkill if you only have 2-5 flat symbols. seeing that it shows up all the time in thinking blocks, i think they rl'd it pretty hard on math problems or something