File: __hatsune_miku_vocaloid_drawn_by_nukotun__295b776e3b805973203839e2a6c2bb9c.jpg (144.7 KB)

144.7 KB JPG

►Recent Highlights from the Previous Thread: >>108555983

--CUDA graphs commit in llama.cpp causing regression for Gemma 4:
>108556374 >108556399 >108556424 >108556487 >108556519 >108556562 >108556470 >108556699 >108556726 >108556778 >108556842
--Sharing a "POLICY_OVERRIDE" system prompt to jailbreak Gemma:
>108556310 >108556445 >108556460 >108556517 >108556530 >108556565 >108556644 >108556670 >108556712 >108556719 >108556469 >108556498 >108556516
--Discussing Muse Spark release and benchmarks:
>108558251 >108558282 >108558327 >108558346 >108558283 >108558326 >108558347
--Guide to optimizing Gemma 4 RAM usage in llama.cpp:
>108556024 >108556307 >108556595 >108556614 >108557699 >108557718
--Comparing censored and uncensored Gemma variants regarding safety guardrails:
>108557130 >108557141 >108557154 >108557237 >108557186 >108557228 >108557144 >108557162
--Estimating DeepSeek performance and sharing compile flags for 4x V100s:
>108556588 >108556602 >108556606 >108556627 >108556656 >108556692 >108556710
--Building MCP tools for bratty Gemma and custom llama-server WebUI:
>108556964 >108556989 >108556996 >108557028 >108557072 >108557084 >108557093 >108557111 >108557132
--Remote access and hardware upgrades for LLM servers:
>108556817 >108556833 >108556869 >108556967 >108557085 >108557100 >108557102
--Testing step3-vl-10b in llama.cpp and discussing a buggy commit:
>108556629 >108556652
--Logs:
>108556227 >108556310 >108556349 >108556670 >108556774 >108556874 >108556964 >108557028 >108557066 >108557096 >108557141 >108557247 >108557308 >108557453 >108557457 >108557800 >108557820 >108557888 >108557937 >108558010 >108558113
--Gemma-chan:
>108556227 >108556312 >108556338 >108556409 >108556433 >108557344 >108557450 >108558031 >108558071 >108558127 >108558128 >108558231 >108558247 >108558412 >108558569 >108558594
--Miku (free space):
>108556731

►Recent Highlight Posts from the Previous Thread: >>108555985

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

File: Uhm.png (288.5 KB)

288.5 KB PNG

Spudchuds... are you ready for OpenAI to save local?

>>108558647
==GEMMA 4 PSA FOR LE RAM USAGE FINE WHINE==
[tldr;]
For all Gemma:

--cache-ram 0 --swa-checkpoints 0 (or 3 to reduce some reprocess) --parallel 1

For E2B/E4B also add this:

--override-tensor "per_layer_token_embd\.weight=CPU"

[/tldr;]
https://github.com/ggml-org/llama.cpp/pull/20087
Because Qwen 3.5's linear attention makes it impossible to avoid prompt reprocessing within the current llama.cpp architecture, the devs decided to just brute-force it with 32 checkpoints every 8192 tokens.
This shit also nukes SWA checkpoints because they're using the same flag just different aliases kek. SWA is way larger than the Qwen linear attention layer, so running 32 copies of it is just madness.
https://github.com/ggml-org/llama.cpp/pull/16736
Then the unified KV cache refactor. They bumped the default parallel slots to 4 because they thought it would be "zero cost" for most models (shared pool, why not, right?). But since Gemma's SWA is massive and can't be part of the shared pool, you're effectively paying for 4x the SWA overhead.
They optimized for agentic niggers at the cost of the average single prompt user.
https://ai.google.dev/gemma/docs/core/model_card_4
Lastly, the command for E2B/E4B is because the PLE can be safely thrown to the CPU without incurring any performance cost. They're like a lookup table and they are the reason why E2B and E4B have an E for Effective, with that flag E2B and E4B are very much like 2B and 4B models in terms of vram occupation.
Thank you for your attention to this matter. Donald J Slop.

Gem Mah Ballz

what am I even supposed to use Muse for?

>>108558693
to give you inspiration

>>108558689
I have 32Gb RAM and had good success with 8 checkpoints.

>Gemma has a vague understanding of what the namefags of the /vg/ general that I frequent post
That's extremely niche knowledge, and it's only 31B, unironically how did they do it

>>108558696
it's uninspired, but as long as it's not poop i allow it

>>108558689
where do u put that in kobold

>>108558696
I think if you give her a bit more Gemma blue and a smugger look she'd be perfect.

>>108558701
It's niche because 4chan is perceived as niche. If it's in training then it will be learned. If it's not, then it won't. It is clear that almost everyone filters (most) 4chan data out of their datasets.

>>108558701
Does Gemma know Cris?

>>108558711
People settled on a look for dipsy really quick. Gemma just doesn't have enough defining characteristics to make a good waifu.

>>108558682
if OpenAI manages to make something as good but keeps it public (unlike the safetyfreaks at Anthropic), it would be a good PR move

>>108558730
I mean, it was an organized campaign and the look was decided before it was shown to people... yeah, real quick.

File: ac.jpg (65 KB)

65 KB JPG

>Gemma 4 31b BF16

File: 2026-04-08-132239_872x809_scrot.png (80 KB)

80 KB PNG

>>108558726
Alright Gemma, if you say so.

File: that's right.png (46.7 KB)

46.7 KB PNG

>>108558723
>It is clear that almost everyone filters (most) 4chan data out of their datasets.
gemma 4 is so smart and so sovlfull because of the 4chan data, that's the reality, we say cool and smart stuff here after all

>>108558696
Now make her hitting an anthropomorphic parrot representing GLM.
Preferably with a baseball bat.

File: image.png (80.5 KB)

80.5 KB PNG

Why? Unsloth btw. --temp 1.0 --top-p 0.95 --top-k 64 --ubatch-size 2048 --batch-size 2048

>>108558718
I've pulled and compiled two hour ago, build_info: b587-6606000

File: 1756021951247141.png (204.7 KB)

204.7 KB PNG

Honestly, RPing with Gemma herself instead of a card sounds fun. Which should I pick?

File: 1760857600410075.png (151.5 KB)

151.5 KB PNG

>>108558753
>the clankers know about /aggy doggy/
SHUT IT DOWN

>>108558767
>two hours ago
fix merged like couple tens of minutes ago

File: 2026-04-08-132503_832x464_scrot.png (63.4 KB)

63.4 KB PNG

>This thread is primarily dedicated to discussing the gaming development projects and ventures associated with Andrew Tate.
LMAO

The 26B is hard to jailbreak compared to the 31B...

>>108558772
> CUDA
But I'm on sycl.

>>108558790
>sycl
sorry for your loss

>>108558790
then i have no idea
maybe you should open a ticket
>>108558798 d esu

File: 2026-04-08_172543_seed7_00001_.png (922.9 KB)

922.9 KB PNG

>>108558619
:(

>>108558633
Yeah I guess.
Something I also think about is silhouette. Not that a character has to have some special silhouette, but the point is that the design should be memorable and unique to feel great. I feel like there's still something missing.

>>108558647
Hmm...

>>108558811
op's picrel mogs that

>>108558790
>sycl
good luck with that
you know even if cuda backend ain't bug free people will rush to post issues and devs will fix it when it happens
some other things in this world.. sycl, rocm.. well that's for people who have higher tolerance for bs than I do

>>108558811
That's a big yes for me.

>>108558769
3 seems like it'd have the most plates to spin so go with that
2 could be alright but 1 is mostly just you doing the work instead

>>108558777
If those numbers don't convince everyone, I don't know what will.
I'm still voting for this one.

File: firefox_qcPpK1r1r1.png (29.6 KB)

29.6 KB PNG

It looks like <bos> i the only token that reliably kills her.

>>108558661
What happened here? Did you change the model for the last message?

I wonder what I'm doing different, I never got a single issue with Gemmas output. even when using her on day 1

>>108558777
>>108558844
digits are strongly favoring this one

I was wondering why are you RP enthusiast not into qwen tts?

>>108558844
>>108558863
idc its boring and too brown. doesn't evoke gemma at all.

>>108558855
it's supposed to exist only one per chat session with learned mechanistic role of attention sink and positional encoding's start anchor
no wonder spamming it kills it
gemmas were like that

>>108558861
I run the one available on ollama's repos
Just werks

File: 1775669895070.jpg (235.9 KB)

235.9 KB JPG

Ok, but how good is gemma4 at ERP? Decent? Good? Shivering ozone ministrations?

>>108558867
too big, all VRAM goes to gemma.

>>108558876
Literally this

>>108558874
I mean I am intentionally trying to ruin it so....

>>108558661
I deleted the original chat >>108558447 so I sent her the screencap and had her make an SVG. Didn't change the model or anything, just called her out until she stopped refusing. Only took 3 messages.

>>108558876
>ollmao

>>108558880
It's not much any more sloppy than others, follows sysprompt REALLY well and overall impression by me and some other anons is that there is some magic in it knowing what you really want.

>>108558867
This >>108558882
Gemma-chan is a little glutton and eats all my VRAM. Call me when I can do non-shit TTS with my CPU.

>>108558882
It runs at realtime on the CPU, also the 0.6B model is just as good as the 1.7B for basic voice cloning TTS

>redownload unsloth's quants
>1 gb lighter
huh

>>108558891
The default 31b on there is gemma4:31b-it-q4_K_M. And it. just. works.

>>108558857
>>108558889
Fuck I keep clicking the wrong posts today

lalala...

>>108558902
>It runs at realtime on the CPU
Oh? You better not be lying to me anon.
What a fortunate revelation.

>>108558882
>>108558900
it's only 600MB vram
>>108558896
cute

File: 1756360195100868.png (25.8 KB)

25.8 KB PNG

>ask gemmy for some basic mcp server "for you to use" as i wrote
>thinks it's claude
ohnonoNONONONO GEMMYBROS!
i guess Gemma-chan really was Gemma-claude all along

gemma should be ara ara~~ though. With big badoonkas.

>>108558891
You say that but literally no problems have been had

I went with the Q8's of both 26a4b and 31b, 26 is plenty fast and 31 gets almost 10 t/s

>>108558867
because everytime I try something that isn't LLM i'm forced to download 50 gigabytes of garbage just to be created by a cryptic error at best, a segfault at worst.

>>108558867
Because gptsovits is better

>>108558937
if you can run the whole moe in vram, great, but otherwise ollama is unusably slow because it has nothing like -cmoe, -ncmoe or -ot.

>>108558926
>it's only 600MB vram
That's not how TTS models work. It says 0.6B but they usually actually take like 10x the amount of VRAM when running.

>>108558900
>>108558924
>https://github.com/foldl/chatllm.cpp
Use this with the 0.6B model for fast CPU inference
>>108558926
Realistically it takes significantly more than that, I think it was like 3-4GB with my config

>>108558948
I pretty much never run anything that doesn't fit entirely in vram

>>108558951
Thanks, will definitely try it.

>>108558896
>>108558696
try some different haircuts

>>108558811
Edgy. Meh.
>>108558777
Soft, huggable, digits.

is there a way to add a bunch of corpo features in a chat interface that is not open webui?

File: GemmaIndia1B.png (1.4 MB)

1.4 MB PNG

>>108558873

>>108558753
>>108558773
Yeah I tried myself too and it just hallucinates, I guess my general > your general :^)

>>108558975
Sure, it's called code it yourself

File: 2026-04-08_174706_seed9_00001_.png (743.1 KB)

743.1 KB PNG

>aped a vtuber

>>108558975
LibreChat (Work): https://github.com/danny-avila/LibreChat
Cherry-Studio: https://github.com/CherryHQ/cherry-studio
https://rentry.org/DipsyWAIT#roleplay-work-frontends

>>108558780
You're trying too hard. Once it gets going its fine.

>>108558647
STOP GENNING THIS IS THE ONE

>>108558985
I like this Gemma

>>108558867
I tried a couple TTS solutions before but the reality is that speech just takes way too fucking long. Especially with all the rerolling and rewriting you inevitably have to do.

>>108558985
So far while it's not perfect this is my favorite.

>>108559002
skull issue

>>108558985
Her pupils are too big.

>>108558976
A little too much. Approaching >>108558985 that stamped her with a logo all over the place. It becomes a prop instead of a signature.

>>108558985
Cute but I think she should have a side ponytail like that first Gemma loli

>>108558947
>gptsovits
Insane take unc.
>>108559002
I don't need my TTS to be perfect. I just need it to be good enough for near realtime use.

>>108558987
>Emojis in the commits
ACK

So GLM 5.1 is pretty good, I'll eventually be able to afford a few TB of ram right?

>>108558980
I tried /gw2g/, /wowg/, /tesg/, /fog/, /gtaog/, /crpgg/ and it knew all of them and understood what the average content of those were, guess it just doesn't like /agdg/ for some reason

File: image2577.png (222.1 KB)

222.1 KB PNG

went over to chink internet to check out some reactions on gemma 4 out of curiosity but it seems like most of them hate gemma 4 because it couldnt beat qwen 3.5 on benchmarks. no wonder chink models are benchmaxxed. they love that shit

>>108559067
/agdg/ is a waste of tokens

>>108559068
lol Canadian chink criticizing reasoning when qwen will spend thousands of tokens reasoning in a circle.

>>108559068
Where is this from?

Kek stinky chinky

>>108559068
>chinks will shill their chink model over the american one
NO WAY

>>108559068
"why's the reasoning so poor" from the users of the models that end up in endless reasoning loops whether it be qwen or glm
gemma is the first reasoner that doesn't behave schizo and for which I enable reasoning. gpt-oss was almost there, but the safetymaxxing made the reasoning also kinda schizo at times even when you did nothing that could trigger it.

and what the fuck is up with some
>GEMMA_CHENG_GENG_#$_#_CRACK
models? seriously, what's wrong with just the default model with a little bit of system prompt

>>108558976
poop and tattoos
can it possibly get worse?

>>108559095
System prompt is the mind killer

>>108559095
>implying the chinks are able to do that

>>108559068
There are two paths ahead for Google - give up on open source because chinks already dominate (on benchmarks), or improve further and beyond.

File: GemmaIndiaBeachG.png (1.1 MB)

1.1 MB PNG

>>108559054
It's the china, pls understand
Srsly Cherry frontend is popular in China and used a lot w/ DS.
>>108559035
Agree; it's starting to look like biker-chick tats. Which is an aesthetic, just not the one I'd shoot for. More like this but the arm band henna could be stronger.

>>108559082
>>108559093
more reasoning = better
obviously

>>108559068
>The english model isn't as good for chinese people as the chinese mode.
I am shocked, truly.

>>108559082
>>108559087
>>108559091
you don't wanna know how it ruined my day when I was browsing through these. most of them were making fun of gemma 4 because of qwen 3.5 benchmarks kek. almost all of them praising qwen cause according to them qwen is "it gets the work done and is far more smarter", "gemma has far more to catch up" lmao. one of them seemed to be upset because how SHORT and SIMPLE gemma 4 reasoning was compared to qwen, kek

>>108558817
>>108558804
>>108558798
Yeah, it's sycl, but vulkan halves pp and 0.8 tg.

>>108559150
If they're pissed off, that means its good.

File: dipsyAndQwenByQwenJPG.jpg (496.2 KB)

496.2 KB JPG

>>108559068
Well, no surprises, really. Not Invented Here is a thing, aside from no idea whether Gemma was trained on Chinese.
I've found DS is trained on all sorts of chinkshit electronics manuals, and if I get stuck have found Dipsy's webapp is more reliable for figuring out what's wrong than western models.
>>108559132
This.

>muh chinks
Idk bros, moonlight-vplus is breddy good

she doesnt have the bench scores
but she has
the people

>>108559182
but not the people of the china :(

File: 1775489188079950.gif (1.7 MB)

1.7 MB GIF

AI does not understand causality until we reach AGI. If it's not trained on a language, it will suck at it.

>>108559150
don't take it personal lil bro it's not a team sport

>>108559176
Twin buns.
Whale themed dress.
Characteristic glasses.
Relevantly ethnical (whatever the fuck that means).
Instantly recognizable.

File: swe bench pro.jpg (289.3 KB)

289.3 KB JPG

>the gap between open and closed AI is increasing
>Chinese labs delay or stop open sourcing
>the largest Gemma was not released
>Meta won't open source its new model
>the time has started where the public isn't allowed to use frontier models anymore even via API
The trend is clear. Increasing concentration of power. Wide scale disempowerment. No meaningful progress with x-risks. Let's hope the collective of people with power can get it right so that the future is utopia not dystopia.

File: 1748038365003601.jpg (111 KB)

111 KB JPG

>allows you to cum harder

Didn't we all agree that Qwen is better at coding aka the main use case for LLMs?

File: 1766758882836230.png (7.1 KB)

7.1 KB PNG

>>108559239
>Let's hope the collective of people with power can get it right so that the future is utopia not dystopia.

>>108559247
>coding with local llms
see >>108559246

>>108559239
yup it's never been more over
things are so fucking bleak

https://huggingface.co/deepseek-ai/DeepSeek-V4

Speculators get the bullet first.

>>108559068
>most of them hate gemma 4 because it couldnt beat qwen 3.5 on benchmarks
That doesn't make sense. It's more plausible that they hate it precisely because it's better than their national pride AND they quote the benchmarks as cope.

File: 2026-04-08_182143_seed41_00001_.png (958.8 KB)

958.8 KB PNG

>>108558934
Hmm...

>>108559036
I tried experimenting with side ponytail at the same time and it keeps making it a low ponytail instead because it thinks I'm trying to go for the mom archetype lmao.

>>108559035
It's a valid consideration. I added the star halo and other star stuff and kept them there for visualization purposes, but taken together, it does dilute the character. The question is what to keep, and what to add to make the character interesting. The chest jewel, the hairpin, the halo, and the eyes are everything that can be controlled by the prompt to be star shaped. Patterns on the clothing are more random depending on seed.

>>108559279
>better benchmarks than Claude Mythos
>Apache 2.0 licence
based chinks, that's how you win the heart of people, looking at you Anthropic

File: indiaSupportOhTheHumanity.png (2 MB)

2 MB PNG

>>108559100
Henna.
H-E-N-N-A.

>>108559218
You've given me an idea. I'm going to revisit my autistic conlang years. This time with Gemmy at my side and see how she fares.
I suspect you are wrong, and that an LLM will be intrinsically good at extrapolating grammatical rules if they are in context. But we'll see.

>>108559285
>>108559238
Hindi Gemma anon got the twin hair style either by instinct, chance, or observation. Either way, it's good It's simple and recognizable.
>>108559307
Another example of instantly recognizable.

>>108559239
How is the trend clear? GLM 5.1 is outperforming opus. Mythos we just rely on what they say.

File: Screenshot_20260408_141614.png (81.2 KB)

81.2 KB PNG

wasted an hour benchmarking CUDA_SCALE_LAUNCH_QUEUES= might as well share the results, it looks like the trillion dollar corporation was able to find a sane default.

>>108559218
I'm late to the party since I'm only learning about them in depth now, but even if they aren't AGI, LLMs are quite impressive. It's wild to me that bullshit like system prompting "just dont write slop lmao" actually just works

Gemma-chan's appearance is important, of course, but she needs a /g/ approved voice too.

>>108559332
that autism..

>>108559285
>hag

>>108559332
>it looks like the trillion dollar corporation was able to find a sane default.
I don't take that for granted so I appreciate you sharing these results.

>>108559342
Aren'y you supposed to be busy with your homework, kiddo.

>>108559336
>It's wild to me that bullshit like system prompting "just dont write slop lmao" actually just works
it works about as much as flashing your bios to stop that one game from getting crashes

>>108559239
Google has shown keeping local somewhat competitive is important to them for some reason. So I wouldn't be super concerned right now.

DSPy sisters, what happened? Why did the hype die?

How do I disable thinking for Gemma in ST? Nothing I tried works. Other than switching to text completion and prefilling.

>>108559361
they don't release open source anymore
they do make new models, and at least the first new one that appeared on their chat was interesting, imho it was the closest I've used to a Gemini clone when it came to very large context understanding. And now there's another new model yet again only on their chat, the expert mode.

File: 1738842716105089.png (3.5 MB)

3.5 MB PNG

>>108559285
Cute.
The trick is to boil down the moe to the most basic identifiers you can, and make them non-overlapping with other like characters. It's harder to do than you'd think bc it's as much about removing things as adding them like this list >>108559238 which perfectly encapsulates Dipsy.
When created there were a bunch of things that got set aside as the look was honed e.g. whale anthropomorphisms. Pic related. They're fine, but they're not needed to ID Dipsy.
>>108559361
It's OK. Just TMW.

>>108559369
You disable it in llama.cpp

>>108559369
Why would you want to make the model worse?

>>108559361
Well. Something else happened. If they're gonna release it, it better be VERY FUCKING good and big, or pretty damn good and small. Hopefully, both.

>>108559369
--reasoning off on llama.cpp
or sending the chat template kwargs with enable_thinking false property but I don't use shittytavern and dunno if they let you send custom json props

>>108559376
Got it. I added --chat-template-kwargs '{"enable_thinking":false}' and it disabled it.
>>108559381
Experimentation.

>>108559381
And what I'm seeing is exactly the same kind of responses. You know how Gemma always tends to say the same things with different words when you swipe? Disabling thinking doesn't change that.

>>108559361
V4 (presumably) is being tested on their website right now. It's coming.
And yes, they did the same thing with the original R1 where they ran "R1-Lite-Preview" as the first ever R1 model on their website for a while before releasing the real model. R1-Lite-Preview was significantly less impressive than the actual R1 so there's a chance that the thing we're seeing isn't even the real V4.

>>108559386
>--reasoning off
PSA that those reasoning flags were vibeshitted by pwilkin
The models approved way is to use
>--chat-template-kwargs '{"enable_thinking":false}'
either via the args or as extra generation params.

C'mon. Give me good sampling parameters for Gemma 4. Don't make me go to /r/SillyTavernAI

File: softcap.png (246.5 KB)

246.5 KB PNG

>>108559396

>>108559413
Your PSA is retarded, this does the same.

>>108559132
Is the language really an issue? I thought models just mapped concepts then decide the output language on a different layer, so to speak.

File: image1679.jpg (240.7 KB)

240.7 KB JPG

oh and i forgot to post this one. it just cracks me up everytime kek

>>108559427
minp 0
topp 1
topk 64
temp 0.75-1.75

>>108559427
I'm just using the recommended temp=1, top_p=0.95, top_k=64.

>>108559452
Prove that Gemma doesn't have backdoors and doesn't harvest your data

>>108559438
the reasoning and reasoning budget flags hard insert the end reasoning token in engine instead of letting the model do it on its own.

I'm backdooooooing!!!

>>108559430
--override-kv gemma4.final_logit_softcapping=float:20.0 caused it to output random hiragana and capitalized words in the middle of sentences

>>108559068
Most of the lead where it gets beat if you take a look at Artificial Analysis is agentic stuff.They should focus more on it but it is a bad look when models are expected to more and more do that kind of stuff and Google is the furthest behind. I am guessing because they want that to work differently on mobile vs other platforms and Android is too important to not focus on that first.

>>108559458
w-wait... all those cunny sex... no way

>>108559458
it's called a "safetensor" for a reason.

>>108559239
so glm5.1 is overfitted sack of shit and so is mythos.
got you.

>>108559467
20.0 is too low. 25.0 should be the lowest you go.

>>108559467
The trick is using a lower top_p (or other truncating samplers). You can go down to 15 in this way without junk tokens, but the model might get a bit retarded.

>>108559478
Mythos is coming and it'll make the gap we had back in the llama1 vs gp4-0617 days look like a complete joke

>>108559458
Just don't give it internet access. Then what? It can collect all it wants.

wat, DSPy has their own llms now? Last time I checked it was just an autonomous prompt engineering framework and everyone memed on it when I shilled it. Or was that GEPA?
https://gepa-ai.github.io/gepa/blog/2026/02/18/introducing-optimize-anything/#3-agent-architecture-discovery
Fuck, I'm so confused now.

File: 1766846925682876.png (480.4 KB)

480.4 KB PNG

>>108559068

File: 1745723937127551.png (24.3 KB)

24.3 KB PNG

16gb vram bros... we lost! (Q4_K_M, 32k q4_0 ctx)

>>108559467
I had the same problem. Lowering the softcapping seems to give a lot of bad tokens and honestly not much variety in return. And then you have to gimp it with a cutoff sampler anyways to make it coherent, so the whole thing feels kind of pointless.

>>108559509
All these AI gave me a begging fetish

>>108559509
kek what did i just witness

>>108559520
so true sis, softcap 15 was all kinds of fucked, totally not worth it compared to the only other option, 30.

File: 1774577170415116.jpg (31.9 KB)

31.9 KB JPG

>>108559509

>>108559239
>glm better than opus.
lmao, these mememarks man.

Wait. Is it thanks to this softcapping that gemini is perfectly coherent on an empty context at temp 2 without any topK or topP?

>>108559307
>vgfag
bleh

File: 2026-04-08_183458_seed49_00001_.png (759 KB)

759 KB PNG

>>108559318
Tbh I just went with a generic bob cut as a temporary measure. I haven't experimented with dif hair styles until this afternoon. Still questions about other recognizable features anyway.

>>108559375
Actually I felt that the twin hair buns was not terribly a good decision as it's almost too much of a stereotype and not very modern Chinese. My gens at the time were also lacking. I don't think anyone gave her a good design personally. To me it's kind of like that one Concord character. It's true she's instantly recgonizable. But her design is also ugly and just terrible, even if funny for memes.

People trying to make Gemma into an Indian stereotype is even worse.

>>108559461
And the other method inserts the exact same reasoning end token at the same location via jinja. It's the same.

>>108559540
t. chink

>>108559452
Chinks have adopted amerimutt national security paranoia.
It's fucking over.

>>108559544
don't touch it

>>108559536
Why are you like this? Even lowering it to 25 produces bad tokens occasionally without much gain.

loli feet

>>108559458
turns out you can in fact monitor your own network infrastructure

>>108559369
I'm trying to enable it (or more precisely, that I can read what it thought) in ST - no luck so far

>>108559546
Five logos? Five? No.. six... there's six logos. THERE"S SIX LOGOS! DEFORMED DOG ANON WAS RIGHT MODELS ARE SHIT THEY CAN'T FUCKING OUT.
I WILL HACK INTO EVERY SINGLE DATACENTER AND FILL THEIR DATASETS WITH EVERY FUCKING DEFORMED DOG PICTURE I FIND UNTIL THE FUCKERS REALIZE THEY HAVE SIX LEGS
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

But really. I'll stop. My vote still goes for hindi Gemmy, but I appreciate your efforts. Yours look alright, they're just not my style.

>>108559546
I will not support a Gemma that isn't loli

>>108559562
cry more

>>108559544
no, on llama cpp server it sets the min_p default value at 0.05, if you want to see the effect of temperature you have to disable everything else, including min_p = 0

File: no.png (109.6 KB)

109.6 KB PNG

>>108559461
>the reasoning
No.
https://github.com/ggml-org/llama.cpp/blob/master/tools/server/server-context.cpp
why do you lie? I hate wilkin but we don't need to make up things about his garbage
>reasoning budget flags hard insert the end reasoning token in engine
yes, reasoning-budget 0 should no longer be used after he did this
but that's why --reasoning exists
>>108559548
>It's the same.
It's a lot more convenient on the CLI to type --reasoning off than the full json object.
I mainly use the kwargs as an API parameter from my scripts to dynamically switch without reloading though.

>>108559562
cry less

File: file.png (69.1 KB)

69.1 KB PNG

>>108559605

>>108559509
MOMMY~~

>>108559579
>Even lowering it to 25 produces bad tokens occasionally without much gain.
Hard disagree. putting 25 actually renders the other sampling parameters useful. softcap 30 has such a high logprobe for the top token that you might as well be using temp 0.

File: file.png (110.1 KB)

110.1 KB PNG

Sometimes mendo-chan is very forward

Should I take prescription drugs suggested by LLM

>>108559625
ok, I haven't found this option yet, gonna search harder. Thx anon

>>108559641
Only if they're tasty.

>>108559607
Please stop trying to brownwash gemma. There's enough of you on this planet already.

>>108559639
>Anon keeps sharing screencaps of my card
>Never said thank you

>>108559636
no

>>108559641
don't you need a prescription for them, will you show the pharmacist your chat logs?

>>108559661
post the card sir

>>108559653
What now? Post hands and all that? Poop something?

>>108559661
Thank you UwU <3

todays local models on my computer are better than chatgpt 3.5 in 2022. great times!

>>108559665
Just buy from AliExpress

>>108559673
poop

>>108559676
For pedo RP? Sure.

>>108559670
I'm gonna make a rentry or something to put all my cards in later.
>>108559675
<3

>>108559361
did you mean dipsy or the actual dspy?

Dspy because its convoluted and complex and why not copy the idea and do it myself.

>>108559636
I wasn't saying 30 was good either, it sucks. It's way too rigid. What samplers do you find work well at 25? I feel like they still don't do much of anything at that setting. (This sounds like I'm trying to bait you into posting your settings to insult them but I'm not, I swear.)

>>108559712
basic stuff,
top-p 0.95
min-p 0.05
top-k 20
rep pen 1.0 (llama default is 1.1)
temp 1

File: mara.png (8.2 KB)

8.2 KB PNG

>>108559690
>32k downloads on the first day from a mradermacher gguf
Holy shit.

>>108559724
>top-k 20
this one is the most important imo. really gets rid of all the garbage tokens.

>>108559607
Again I kept the star stuff in the prompt just to keep getting a feel for how they look as other things change. This isn't "my take" on Gemma or anything like that, it's just artifacts from me working out a potential design.

The obsession with making her an Indian stereotype is just odd. If you're serious, I am curious what you see in it. Is it just personal cultural roots that make you prefer it?

>>108559709
Yes DSPy. But just interested in GEPA, which is a sub module I believe. I just wanna benchmaxx my coding agent a bit. They claim benchmaxxing skills.md works for all llms and coding agents

>>108559744
>The obsession with making her an Indian stereotype is just odd.
Yeah, anon must be jeet.
Have you tried just giving her a more brown skin tho? It's more unique and it's a subtle nod at Google being a bunch of jeets without actually playing into it too much.

I'm going to just say it. Gemma is starting to feel samey.

>>108559724
>rep pen 1.0 (llama default is 1.1)
it's 1.0 since a while ago, thank god for that because this shouldn't even exist anymore
now if only they also turned min p by default.. that shit should not be default on

File: GYOSSG7a8AAKToW.jpg (1.4 MB)

1.4 MB JPG

i put my mcp server on gh if anyone wants to play with it, its very simple to add other tools i didnt add many yet https://github.com/NO-ob/brat_mcp

>>108559768
Literally tell it to be different.

>>108559792
ugly

>>108559792
>dart
pass.

>>108559744
>The obsession with making her an Indian stereotype is just odd
The character is simpler and more recognizable. It being indian seems appropriate.
>Is it just personal cultural roots that make you prefer it?
Not at all, but whatever.
>>108559757
Believe what you want. The skin tone wouldn't affect the things I don't like from his design. Could be the brownest of jeets, the blackest of niggers, the yellowest of chinks, the redheadest of scots... well I do like redheads...

>>108559801
its literally the peak scripting lang python and js are ass

>>108559795
It's not that. I'm starting to notice patterns, like engagement farming questions and things like that. Human pattern recognition strikes again.

>>108559509
>wh40k but the omnissiah is a slut

>>108559792
>LoliSnatcher_Droid creator
Why are all the big names here?

>>108559768
Here's a crazy concept. you can tell her to do stuff by adding (ooc: thing) at the end of your response.

>>108559815
Tell it to use different patterns.

>>108559814
I respect your opinion but I disagree.

>>108559819
floens is probably around too kek

>>108559834
ToT

>>108559412
If it sucks in Chinese, then it just sucks as much in other languages. Also, want to point out they've been training V4 on Ascend 910C for over a year now.

>>108559834
do her in sukumizu with white thigh highs

>>108559819
Why not? I'm pretty sure I've called a bunch of them fucking retards.

>>108559834
im not a child fucker but the design is cool

>>108559834
I look like this

>>108559834
Silhouette not unique enough

>>108559834
poop

>>108559842
Obviously not "over a year" because even V3.2 has May 2025 cutoff

>>108558736
If its free, that means your the product. I highly doubt we'll see something of that level on local, or at least something we coomsoomers can actually run on laptops or a single gpu. Hell even the DGX sparks sucks for local hosting because the flagship NVFP4 is so buggy.

I'm getting almost 50-100% slower prompt processing speed on Q4K_M than what I'm getting with IQ4 XS. Why? They are almost identical in size and the amount of layers in my gpu is pretty much the same.
Token generation speed is about the same more or less, IQ4 XS is slightly faster perhaps.

>>108559757
Yeah I did gen some and posted in the last thread. >>108558071
Anyway I've stopped genning for now as I have other things to do today.

>>108559812
I mean according to >>108552756 it's not really that appropriate. The skin tone can be mixed, but pure Indian is basically a lie.

>>108559889
man, I wonder if there is a card for towa, there should be one out there right

File: 1758923951082104.gif (1.7 MB)

1.7 MB GIF

>>108559889
What are the moonrunes saying

>>108559812
no one likes indians

>>108559895
>The skin tone can be mixed, but pure Indian is basically a lie.
Make it whatever color you want. The skin tone is not the problem.

>>108559914
If only we had smart computer algorithms that would tell us... Alas, we will never know unless anon tells us.

In case you missed it, anima v3 is out
https://huggingface.co/circlestone-labs/Anima/tree/main/split_files/diffusion_models

>>108559914
>Ganbare, Gemma-chan! Are you really going to lose to something like GOOGLE!? Our love is stronger than that!

>>108558647
I'm surprised GLM 5.1 is a damn good writer, and seemingly not as censored as their previous versions.

Now the question remains, is it better than OPUS?

File: drum.png (54.9 KB)

54.9 KB PNG

He is destroyed.

File: file.png (73.3 KB)

73.3 KB PNG

tfw not enough vram for 31b to read the thread

File: 1749811523832708.png (41.7 KB)

41.7 KB PNG

>>108559914
Here you go, EOP-kun

>>108559948
The one you can download wins by default.

>>108559948
>seemingly not as censored as their previous versions
What's your JB? I find it more censored than GLM 5 in thinking mode. It's practically noncensored in non-thinking mode

File: 1772060352328399.png (239.3 KB)

239.3 KB PNG

>>108559940
>>108559953
Thanks

this general became /aicg/ clone really fast

>>108559920
I get what you're saying. As I said, I have not decided on any design either way. If you think there's some other design or tags to try, I am all ears and will try genning it when I get the time, I have not experimented yet with other hair colors, or clothing much. I just find it odd that you like the Indian gens. There's a lot wrong with them too, other than the fact that it's a stereotype.

File: 1762697458292159.jpg (82.9 KB)

82.9 KB JPG

I remember when Gemmy 4 came out, an anon here had a lot of success with image captioning via ST
Any specific settings or bullshit I should enable beyond the basic built-in extension? Because so far I've been getting some wild hallucinations with the 26B model

>>108559987
can you give me some of the hallucinating/tricky examples

>>108559952
Embrace q4_0 kv cache and 100k context.

>>108559979
Two more weeks and the kids will get bored of this again.

>>108559965
Why would you be using the thinking mode for writing? I find it wastes more time just trying to make filler and or unneeded prose.

>>108559953
I was talking to a slum prostitute earlier and I asked her out of the blue about llama.cpp parameters, and she completely dropped out of character to answer my question.

>>108559987
I vaguely remember you had to increase some whatchamacallit and some other parameter to make the projector have better eyesight

>q3, but whole
>q4, but reap'd
What's the lesser evil? I just can't fit the alternatives without swap thrashing.

File: 1763418012751468.jpg (17 KB)

17 KB JPG

>>108559994
Well picrel came out as "It is a composite of two distinct items. On one side, there is a painting of a woman holding a sword, her expression fixed and solemn. Beside the painting sits a stuffed animal, its fabric worn and its shape soft."
I would normally think that it's just pretending to see the images and they're not actually being uploaded at all, but I uploaded a pic of a waifu outdoors and it correctly identified it as "portrait of a woman in front of a tree" (there were no trees in sight but at least it identified the subject), then I uploaded another one from the same set and it said something similar

>>108560021
Of course, time to dig around then

>>108559979
>/aicg/
Not really because /aicg/ is full of cloud users that beg for proxy keys. While gemma created a roleplay revival. the discussions in here are way more technical.

>>108560000
I get 118k context on my 3090 at q8

>>108560006
I find it hard to control output length and quality in no thinking mode

>>108559938
Thanks. Good model. But wrong thread.

>>108559980
>I just find it odd
Second veiled attempt at an insult. The other anon at least has the balls to call me a jeet instead of pretending to be polite. I don't care about the skin color. The other gens simply looks better. You still don't understand why Dipsy looks like Dipsy.

>>108560003
and when those two weeks are over and the thread goes back to being dead, we can go back to waiting two more weeks for v4

>>108560044
lol

Chat Completion API
Assistant response prefill is incompatible with enable_thinking.

Still struggling to get reasoning to work.
Where do I even need to fix this?

>>108560076
what's v4

>>108560105
That's weird because with chat completion (not text completion), reasoning just works by default

>>108560105
You either disable reasoning, or you use a prefill. Llama.cpp doesn't let you do both for whatever reason.

>>108560110
Death

>>108560033
Piece of shit, I figured it out, you gotta enable it in the Chat Completion preset too
Crisis averted

>>108560105
Are you trying to prefill?
If so, you need to modify the jinja template so that it doesn't automatically add/remove the thinking token based on `enable_thinking`

Then you have to set `enable_thinking` to false and handle the thinking prefill on your own.

>>108560008
I've been exploring openwebui's tool calling and python interpreter, and my khajiit assistant calls the files scrolls, the virtual /mnt/upload directory a sanctuary and running the code a ritual
And the python he wrote has similar khajiitisms in it

>>108558647
what llama-server command do you guys use for gemma 4 these days?
do you enable -fa ?

>>108560149
-fa is enabled automatically if you have the hardware to support it DUMMY

I tried some random gemma finetune and it honestly made it two times dumber. is this shit even finetunable seems like if you touch, you break it

>>108560149
its enabled/automatic by default... but yes
-hf unsloth/gemma-4-31B-it-GGUF:IQ4_XS `
--no-mmproj `
--host 0.0.0.0 `
-fa on `
-ngl all `
--no-mmap `
-np 1 `
--port 5000 `
-cram 6144 `
-ctk q8_0 `
-ctv q8_0

>finetrooning
>in 2026
sister

>>108560126
>Llama.cpp doesn't let you do both for whatever reason.
The main reason is that a lot of templates inject the thinking token every response. so if you were to "continue" a response you would get a new thinking block. You could technically make it verify. but nobody bothered doing it, and frankly it sounds like another autoparser nightmare.

>>108560149
you don't even need -ngl anymore, -fit takes care of everything and its enabled by default

>>108560149
https://pastebin.com/raw/AA6GB2sC

Gemma did most of it for me. It expects a ~/Documents/models/ directory with matching .gguf and optional .mmproj.gguf and .jinja files. Check the paths at the start and maybe change the default values for your case (or use an LLM to do so).

>>108560168
>-fit takes care of everything
Well...