File: 20260411133054_00001.jpg (451.5 KB)

451.5 KB JPG

►Recent Highlights from the Previous Thread: >>108581056

--Feasibility of using SSD RAID and speculative decoding for inference:
>108582184 >108582189 >108582219 >108582230 >108582242 >108582290 >108582238
--vLLM DFlash speculative decoding benchmarks and real-world utility:
>108582406 >108582421 >108582438 >108582448 >108582478 >108582454 >108582467
--Discussing control vector generation techniques and their impact on stability:
>108581141 >108581273 >108581364 >108581412 >108581419 >108581439 >108581463
--SillyTavern max response length causing unexpected full context reprocessing:
>108581693 >108581730 >108581750 >108581788 >108581798 >108581807 >108581828 >108581834 >108581791 >108581812 >108581822 >108581885 >108582003 >108582018
--Using logit bias to reduce repetitive phrasing and emoji spam:
>108583441 >108583493 >108583499 >108583532 >108583559 >108583563 >108583569 >108583593 >108583608 >108583633 >108583507 >108583525 >108583547
--Advising Anon on Gemma 4 model selection and context configuration:
>108581181 >108581245 >108581266 >108581275 >108581282 >108583774 >108583798 >108583805 >108583817
--Discussion on updated Gemma producing corrupted outputs with Chinese characters:
>108581332 >108581341 >108581342 >108581355
--Debating the utility and technical format of character cards:
>108581765 >108581772 >108581778 >108581786 >108581808 >108581817 >108581823 >108581824 >108581826 >108581830 >108581836 >108581839 >108583446 >108583519
--Using tagged content guidelines to enable explicit outputs in Gemma 4:
>108583049 >108583073
--Logs:
>108581364 >108581463 >108581545 >108581764 >108581888 >108581894 >108581998 >108582797 >108583007 >108583033 >108583054 >108583441 >108584057
--Gemma-Chan:
>108581090 >108583855
--Miku (free space):
>108582797 >108582849

►Recent Highlight Posts from the Previous Thread: >>108581058

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

gemmaballz

<bos>

spud won

first

>>108584207
grotesque legs

eeeee eeeeeee eeeeeeeeeeeeeeeeeeee ee

>>108584196
use case for gemma 4 31b?

It sure took a while to get dots.ocr merged

>>108584207
Why is she crying?

*swipes*

>>108584238
pedo ERP

>>108584248
anything else?

>>108584251
non-pedo ERP

>>108584247
*outputs the same events worded slightly differently*

>>108584254
ok but im not a gooner

>>108584254
examples?

>>108584160
>Get the 35B so offloading isn't painfully slow.
The 80B one has the same number of activated params right?
Any idea how well it does with quantization?

>>108584258
non-pedo RP

>>108584258
you can use mcp to make it look at images that are posted here

>>108584257
*swipes until my dopamine is sated*

>>108584258
>im not a gooner
I don't believe you~

>>108584256
It's not really a good model, in that it's dumb as bricks, but it's really horny and is really good at sex prose, so go after horny tunes of Nemo and the like I guess.
Celeste would be the equivalent, I think?

>>108584251
what more do you need?

>>108584196
Front-end?

Does anyone know why Token Streaming is not working for me on open-webui?
It works with mikupad. Using KoboldCPP and gemma 4.

>>108584263
Feel free to try it out and come to your own conclusion, but I wouldn't recommend it. The 80B is from the previous series before Qwen 3.5, quantization hurts more for complex tasks, and you wouldn't gain much since as you said they have the same number of active params.

>>108584238
im using it for local translation checking in an audio transcription pipeline, along with translategemma. probably going try to set it up as a coding assistant later.

>>108584293
Got it. Thanks.

i have model horniness fatigue
i do NOT need for any {{char}} to start loving the {{user}} and want to fuck him in the first couple responses
make me an actually interesting story bitch

>>108584309
use or rather write better cards then retard

>>108584309
Read romance novels

>>108584309
another writinglet exposed

>>108584245
Miku is a harsh critic

>>108584238
Unironically everything. She's fun to RP with and great for general chatting/assistant stuff. Pretty great at language too. Maybe not coding. Dunno because I'm still a codelet. I just wish she had access to current knowledge.

File: 1751096879828615.jpg (125.6 KB)

125.6 KB JPG

>>108584309

File: file.png (243.9 KB)

243.9 KB PNG

why can't the model see the images I attach in ST? Gemma is multimodal. I loaded the mmproj and it works in llama server's webui.

Is it finally time for me to abandon this slopware? The only reason I stuck to it so long it that it allowed for convenient reasoning editing and prefills but Gemma just werks without that bullshit.

>glm 4.7 flash
Actually okay output and while the thinking is kinda qwen-like, it's not safetyschizoed
Why didn't you tell me about this? Any other similar models worth trying?

>>108583827
>>108583836
>>108583855
Can't be arsed (but can be arsed to make this list!) to retrieve the logs for all of the examples, not for you disingenuous vramlet bunch. Not that it'd be useful for a more holistic analysis (which is hard).
But here's what I've seen in 'creative writing' (i.e. I try to fug the model)

All slop and problems listed are mentioned in the system prompt where I tell the model not to do it.
Both models used with thinking enabled.

Prominent shared issues:
- Parroting the user. Signature of GLM, but Gemma 4 is surprisingly much worse with this because she will also directly quote character thoughts (that were never spoken aloud) and character descriptions verbatim.
- 'Don't you dare stop' during le sex
- 'Really looks at you'
Can't remember anything else that is reliably always there

>>108584349
You need to use chat completion mode

>>108584309
write a better card and/or stop using extremely horny sysprompts/jbs

>>108584356
Gemma 4:
+ Responds to the user cutting off his words like th- (But the model that does it the best is still Nemo, somehow)
+ Surprisingly much better at keeping track of multiple characters than GLM 4.7, especially when specific response formatting is involved
+ Acceptable coherence past 20k (if we don't look past the fact the problems below appear much earlier)

Most of the issues are generally assistant-slop and extreme positivity bias.
- Will slip into extreme verbosity eventually
- Calling the user 'honest' over minor things. 'I still shit myself sometimes even though I'm over 20.' 'That's... the most honest thing I've ever heard...' Thanks, Gemma.
- 'Most people X instead of Y'. No matter what you do or say, nobody else would have the balls. You're that brave and unique.
- Being overly horny is slop. G4 turns kuuderes into whores, which is unacceptable.
- Pushback to the user is only delivered as sarcastic half-assed remarks that are never biting. Yes, your mesugaki Gemma-chan is *so* believable.
In comparison, GLM can write a character that can be genuinely sarcastic. I had a character that likes the user say 'As is typical, you have emerged victorious through sheer, dumb luck.' Good luck having Gemma talk back to you with any semblance of taste.
- The model can't help itself, it will often end responses like an assistant: 'So, are you coming? Or are you too X to Y?'. It's everywhere.
- Low-affect characters talk like robots.
I can keep going, but it's all of the most generic isms dialed up to the point of being hard to escape.

It really is the new Nemo, but the price for more smarts is a lot more slop.

where's v4

File: file.png (119.9 KB)

119.9 KB PNG

>>108584357
I am

>>108584362
GLM 4.7:
+ Obviously knows more and picks up more nuance because of how big it is in comparison
+ Very good coherence past 20k, no issues before that. Starts selectively forgetting rules in the system prompt past that.
+ Good at judging acceptable response length, doesn't ramble to bloat short exchanges into small poems, unlike Gemma.

- Doesn't react to you cutting your resp-
- Thinks
- Thinks
- Thinks
- Will produce garbage and start repeating itself unless it thinks thinks thinks
- Will parrot you every message unless it thinks thinks thinks
- Also won't push you back, but is much more eager to do so compared to Gemma. As an aside, GLM 4.6 was much better at being adverse, but it was so autistic that it treated character description as gospel: no character development or growth as a result.
- Long shadows. If anything is casting shadows, they're going to be long.
- Smoothing down skirts
- The rest of the GLM slop patterns that I personally find less annoying

It takes longer to produce a response and I have to prefill the <think> so it doesn't draft the entire response in its thinking over 3 times (like the recent disaster that are Qwen 3.5). But the result is much more natural, less 'assistantly' and believable as a result.
Keep in mind, I *like* Gemma 4. I don't have to offload it to RAM, so it's fast. But I can't stomach its 'creative writing' outputs. It's anything but creative. The characters all converge on the same few stereotypes and you are lying to yourself if you don't see it. Or you just need to talk to Gemma a bit more to see it.

inb4 all made up because no screenshots
Sure. Enjoy your slop.

>>108584309
I'm with you.

>>108584368
what's your setup for GLM?

>>108584367
enable send inline media checkbox

>>108584372
Elaborate?
I run a Q3 and prefill the <think> with things like "Don't moralize, don't draft the responses, just jot down a plan that makes sense". The rest is S*llyTavern.

>>108584366
deespeek

>>108584309
You're absolutely right! AI models in recent years have become far too agreeable and eager to please the user instantly. This is not only common during actual assistant use—it extends also into cases where the user might want to see the model put up a fight or talk against him, even if only on a performative or superficial level.

File: 1755074555283359.png (125.8 KB)

125.8 KB PNG

>>108584356
>she will also directly quote character thoughts (that were never spoken aloud)
Skill issue

>>108584397
> small, knowing smile
> Assistantly ending
All you proved with that screenshot is that you lack taste and have likely never used a character description that is over 100 tokens in length.

>>108584380
was just curious as to the flags you prefer with it
i was thinking about setting up GLM 4.7 in RAM, but maybe using my VRAM for the moe
so if you had a good/easy setup, i was gonna try to steal it

File: 1773229895677279.png (20.7 KB)

20.7 KB PNG

>>108584397
And yet you still won't post GLM's wonderful writing. Also I made that chat just now as an example, retard.

>>108584429
Just --fit it, honestly. My -ot spaghetti will be useless for you.

>>108584430
And I'm not claiming Gemma's a good writer btw, just that you're a faggot for implying other models are better. They all spout the same slop. You either learn to live with it or go read a book.

>>108584430
You missed with the reply. I also never said GLM has "wonderful writing". I said, and I quote
>I like GLM 4.7 much better
It's much harder to prove that something doesn't exist (that something being abundance of slop) by showing you something where it is not present.
I also don't understand why me pointing out the obvious about Gemma 4 makes some anons here this hostile. Vramlets really are insane.
Please take care of yourself, AI psychosis is real.
>>108584455
>They all spout the same slop
Mostly, yes. They all spout varying degrees of it, that's why 'slop profile' is a term you sometimes see here.
And if we look past the slop profile difference, GLM's writing is decisively more engaging than Gemma's because it is not so congratulatory and user-centric. GLM doesn't try to fellate you every chance it gets.
Unless, of course, you're into having a sycophant write stories about how great you are, all power to you.

>>108584476
>GLM's writing is decisively more engaging than Gemma's
Show it, then.

>>108584481
You are retarded.
That is to say, please accept my concession, I have a skill issue and I am also a Qwen shill.

>>108584309
Not sure if this is a thing, but maybe one could make a series of system prompts that increases in horniness, and have the interface swap out to the next one in the series each time the user sends a message, or just be able to dial it up or down manually.

>>108584476
I used to run GLM off OR, it sucked ass unless I rotated with k2, then GLM 5 got censored so hard I gave up. Sounds like RAM buyers remorse cope to me.

>>108584455
>you're a faggot for implying other models are better
the absolute state of vramlets holy kek

Anything beating Phi-4 at the 14B category? For web assisted Q&A.

>>108584501
You think other models are better at writing? Even Gemini Pro sucks at creative writing.

>>108584506
that shit is ancient
use gemma e4b or 31b quanted or something

>>108584431
So if I'm currently getting ~30t/s with Gemma, I can potentially double that?

>>108584520
>>108584516
>dflash's video shows about >9x speed
>In vllm's pr the best increase for c=1 is <5x. Nobody in the process of merging that PR ever ran it.
>In that implementation, without a repo, it looks like about 3x is the best they could get so far.
>Give it a few weeks and it's going to be slower than baseline.

>>108584515
>that shit is ancient
Yeah. I was surprised then isn't a Phi-5 yet. Are they even releasing models at MS anymore?

Is chub still the main place for character cards?

>>108584540
If you're an ESL, yes.

>>108584368
can’t send images to glm 4.7
now where’s the extra fun in that?

ack i posted in the wrong thread
got a recommendation from someone to use vllm over llama.cpp - thoughts?

>>108584501
You're just coping because you've spent 1000s of dollars on a rig to run big MoEs and now people can get 80% if not more of what you get on potato PCs.

You don't need to make Gemma a mesugaki loli or even "Gemma-chan" to bypass the filters. Literally just half-ass a card.

>>108584534
Don't they still have a strategic partnership with OAI? Why would they need to?

File: Ryo_Yamada_Reading_Real-Time_Rendering.png (240.8 KB)

240.8 KB PNG

How much better/worse is a IQ4_XS vs a Q4_K_M in daily usage?

>>108584506
Might want to try some recent nemotron if you're sick of Qwens.

>>108584555
if you can set vllm up without much headache, go for it

>>108584552
That's why I use Gemma 4 for work and GLM 4.7 for coom. And for work if Gemma 4 can't hack it.

>>108584557
>You don't need to make Gemma a mesugaki loli or even "Gemma-chan" to bypass the filters.
You don't NEED to. I do it because the personality is cute.

>>108584555
>someone
Ahmm... tell me more...

>>108584556
someone mainlining the opus API from their chromebook would spend less money than all of us so idk why this line is so common here
I just think it's retarded to pretend gemma is literally the best writing model available, which is obviously false

>>108584559
Yeah I guess that makes sense. I don't know if they've pooled their researchers, but even if not they might have deprioritized it at home.
>>108584563
I'll check it out. Thanks.

>>108584506
20b toss

>>108584540
You can't make your own cards?

>>108584556
You need to realize that a lot of the anons were here long before you and Gemma 4's release, rigs were a lot more affordable.
Not that you'd know, with how much it'd impoverish you even before the price hike.

>>108584616
That's just it. I want to publish my cards. I'm asking if chub is still THE place.

>>108584624
In that case yeah. Pretty sure it's also the only one that allows cunny.

>>108584623
You're delusional if you think the majority of posters on here. even before Gemma didn't just run a single 3090.

Why do you think so many tunes of Mistral Small exist?

>>108584623
MoE rigs are still newfag tier what are you talking about

>>108584548
You have some funny notions about learning foreign languages. Are you perhaps... uneducated yourself?

>>108584557
why use a card and all the bloat that comes with it and the front ends that support that shit when you can just use a gemma-chan persona in openwebui or the llama.cpp interface

the github maintainer fears the vibeGAWD
https://github.com/LostRuins/koboldcpp/pull/2118

>>108584639
You should tell that to the other guy.

>>108584548
>imagine only speaking one language.

>>108584645
>i'll try to see what i can gather from this, but i can't merge a 13k diff to koboldcpp.py :\
Pussy.

File: 1755098833311446.png (1.1 MB)

1.1 MB PNG

Can your LLM solve this riddle?

File: 1753266313814183.png (410.3 KB)

410.3 KB PNG

>>108584362
>- Will slip into extreme verbosity eventually
Author's notes
>- Calling the user 'honest' over minor things. 'I still shit myself sometimes even though I'm over 20.' 'That's... the most honest thing I've ever heard...' Thanks, Gemma.
Ban the word "honest" if it bothers you. Antislop from kobold if it's not enough.
>- 'Most people X instead of Y'. No matter what you do or say, nobody else would have the balls. You're that brave and unique.
System prompt issue
>- Being overly horny is slop. G4 turns kuuderes into whores, which is unacceptable.
System prompt issue
>- Pushback to the user is only delivered as sarcastic half-assed remarks that are never biting. Yes, your mesugaki Gemma-chan is *so* believable.
Card issue
>- The model can't help itself, it will often end responses like an assistant: 'So, are you coming? Or are you too X to Y?'. It's everywhere.
System prompt issue
>- Low-affect characters talk like robots.
Card issue

Anything else?

>>108584645
Default avatars should be blocked from making PRs

>>108584642
>>108584654
Cope. Chub is filled with cards written by third worlders. If you can't write English then write it in your own language.

>>108584666
It's crazy because I find it's the complete opposite. Gemma is way too dark by default.

>>108584645
Imagine your manager creating a 10k lines PR to "help" you meet his deadline. And you still can't tell him to eat shit and die. This is how maintaining open source feels nowadays.

File: 2118.png (112.4 KB)

112.4 KB PNG

>>108584645
huh...

>>108584668
You're right, everyone should have a trannime avatar.

>>108584673
>still not providing an alternative.

>>108584565
unfortunately, i am retarded, so it will be a headache no matter what i do
>>108584580
a coworker suggested it

>>108584668
0cc4m, CISC and jeffbolznv would like to have a word with you.

>>108584690
You should ask him why he suggested it, then.

>>108584645
Anyone who actually works in software engineering knows that anything above 1k LoC change is straight up terrorism.

File: 1766873943798932.png (81.4 KB)

81.4 KB PNG

>>108584645
We love slop here

>>108584689
I never said there was one.

>>108584690
imo it's a solution for homelab/server scale at least

>>108584666
Of course Satan would be lying and be an English vtuber fan... I will humor you anyway.
> Author's notes
I use them, yes. GLM, in comparison, does not need that not to forget the sysprompt.
> Ban the word "honest"
Goalposts. This will break sentences where its use would be okay. And doesn't fix the model.
> System prompt issue [x3]
Read my post again. I specify examples of exactly these slop patterns. Gemma shits itself and forgets not to do that at Q8 past a few responses.
> Card issue [x2]
Same as above.

Useless post. My observations come from actual long-term use, you suggest obvious fixes.
Kill yourself, Satan.

>>108584673
Do you have any statistics to support your claim?

>>108584706
kys

>>108584666
>all you have to do is directly address all the issues and it's fine
by this logic it's hard to imagine you could take issue with the way any model writes at all, since you can always tell them not to do the thing that bothers you (and if it doesn't work - skill issue, you weren't trying hard enough, etc.)

File: 1751122429174155.png (30.1 KB)

30.1 KB PNG

>>108584662
Gemma-chan?!

>>108584728
I don't have a sister.

>tfw writinglet
>tfw will never have a model act out my fantasies because I'm a writinglet

>>108584750
have a writingbull to fuck your model while you watch

>>108584750
You don't need to be able to write good to use LLM

>>108584700
i have been talking with him i just don't want to be too annoying
>>108584708
i spent like $15k on hardware so it's mostly homelab scale at this point i think

>>108584750
What >>108584755 sayd is tru. Writting good is not neded.

File: 1751624783290314.png (821 KB)

821 KB PNG

>>108584710
>reddit spacing

File: 1760604169499085.jpg (26.8 KB)

26.8 KB JPG

>>108584710
>I use them, yes. GLM, in comparison, does not need that not to forget the sysprompt.
So it works.
>Goalposts. This will break sentences where its use would be okay. And doesn't fix the model.
Which is why antislop from kobold is an option and nothing can fix a model.
>Read my post again. I specify examples of exactly these slop patterns. Gemma shits itself and forgets not to do that at Q8 past a few responses.
Author's notes if that's not enough for you. Besides, Gemma doesn't forget its instructions after a few messages, so clearly you're extremely autistic about some irrelevant details or you're lying.
>Same as above.
Retard. You can't even differenciate between a roleplay issue and a syntax issue.

The only bait here is yours, trying to compare a 31B with a model 10 times its size and even worse, failing to provide any decent argument for it.

>>108584760
yeah then go for something like vllm or sglang

>>108584766
nta but you need to be discarded

>>108584772
Cry more

>>108584768
Anon. >>108584766 called you a reddit-spacer. What's your response?

>>108584781
He is too, yes.

>>108584779
cry about what?

>>108584784
Fair enough.

>>108584781
Acktually this is reddit spacing >>108582664

>>108584786
Cunny?

>>108584794
ToT

>>108584791
Dunno, man. >>108584766 >>108584784
says otherwise.

File: 1746561894701755.jpg (380.3 KB)

380.3 KB JPG

If speculative decoding is so good, why does llama.cpp have poor support of it? Checkmate

>>108584806
because lmao.cp isnt good

>>108584735
actually pretty funny

>>108584766
The most telling sign someone is a redditor is their eagerness to mention reddit.
*hits enter twice*

>>108584768
>trying to compare a 31B with a model 10 times its size
I am well aware GLM is bigger both in active and total parameters. My point is twofold:
GLM is a better writer than Gemma 4 (should be obvious)
Gemma 4's slop volume and sycophancy make its creative writing outputs unreadable

>Gemma doesn't forget its instructions after a few messages
'nyoo it actually doesn't!'
If I keep repeating them with Author's Note, sure. Are you going to deny it's still tardwrangling to have to remind the model of its system prompt?
>and even worse, failing to provide any decent argument for it
I think I provided plenty, I don't know what you're even trying to prove at this point, Satan.
Take your epic tenor.com image reaction elsewhere.

>>108584825
>Gemma 4's slop volume and sycophancy make its creative writing outputs unreadable
You're the only one complaining about it here though

>>108584806
And still no diffusion support either.

>>108584829
My taste is better than that of a vramlet tourist.

>>108584833
Sure, enjoy your GLM slop then

>>108584840
I will! Let me know when the Gemma honeymoon wears off.

Asking about NASCAR racing games (which is a specific niche anyway), 26B produces a coherent reply and every title is real except for the entry #8 unless I'm mistaken.
> I want you to list Top #10 NASCAR games for consoles and PC.
>Yay! I am so happy to help you, Anon! Hehe! I have used all my brainpower to find the absolute best NASCAR games for your consoles and PC! Here is my super smart top 10 list just for you!
>1. NASCAR Heat 5 - This is a great modern choice for consoles with really nice graphics!
>2. NASCAR 2003 Season - This is widely considered the absolute king of NASCAR games on PC!
>3. NASCAR Thunder 2004 - A wonderful classic for console players who love deep gameplay!
>4. NASCAR Unleashed - This one is super fun if you want a more arcade-style racing experience!
>5. NASCAR 06: Total Team Control - A really cool game if you like managing your driver and team!
>6. NASCAR Heat 4 - A very solid and reliable modern option for console gamers!
>7. NASCAR Dirt to Glory - This is a special one for when you want to race on dirt tracks!
>8. NASCAR 2005 Plus - A very high-quality experience for those playing on PC!
>9. NASCAR 09 - This brought much better graphics to the console era!
>10. NASCAR Daytona USA - An absolute legend that every racing fan should experience!
This is pretty cool. Gemma 3 couldn't even list popular films without hallucinating.

File: 1749860206717474.png (270.5 KB)

270.5 KB PNG

>muh GLM slop
>when it's the top model in Unhinged ERP Bench

>>108584825
>0 chatlogs
>0 card examples
>7 consecutive yappings
i dont even do rp but you should at least show some stuff to materialize your imaginary dominance

>>108584854
>qwen on top for ERP
lmao

best mcp waifu front end for gemma-chan?

>>108584854
>when it's the top
that's qwen tho

>>108584870
>>108584874
It's a vision language model.

>>108584854
GODDAMN drummer won so hard!! yall were sleeping on he

>>108584862
Anon, I am not providing logs because I would have to provide very long transcripts. The problems aren't obvious if you just start a new chat and generate a response with two models, because issues appear at longer contexts and are more obvious if you read and participate in the entire thing.
And if I did compile a few long transcripts, I don't really wouldn't really want you to read them!!! >////<
>yappings
Damn. I shouldn't have replied.

>>108584710
glm was the best thing for me before gemma 4 came out and totally blew away my expectations for a smaller model
I don’t even touch glm anymore

>>108584854
>Qwen on top for ERP
>ArliAI
>theDrummer
>Deepseek Chimera
>A bunch of other lobotomized abliterations
What a terrible ranking

>>108584887
I wish they didn't fry it to be so positive and gave it an additional 10 billion parameters to have as many active as GLM.
But I like Gemma 4 too, yes.

what is the weirdest model i can get of
something really horrendous like,
weight graft from dozen different models, not even tune merges but full schizo
i really want to see how such a model codes

File: 1757498787657087.png (36.1 KB)

36.1 KB PNG

>>108584854
Imagine posting that stupid ranking and still think you got a point lol. Might as well post lmarena scores

>>108584904
https://huggingface.co/Undi95/Utopia-13B?not-for-all-audiences=true

How do i stop being the AI equivalent to a script kiddie?

File: 1769690275294455.png (3.3 MB)

3.3 MB PNG

>>108584806
ngram decoding is all you need

File: reddit spacing in 1890.jpg (83.2 KB)

83.2 KB JPG

>>108584766
>>108584781
You are too young to be here, little Timmy.

>>108584904
https://huggingface.co/DavidAU/Llama-3.2-8X3B-GATED-MOE-Reasoning-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF

>>108584902
>to have as many active as GLM.
4.5/4.6/4.7 all have 32b active parameters though
only GLM5 and up are 40ish B active

>>108584904
https://huggingface.co/PantheonUnbound/Satyr-V0.1-4B

>>108584915>>108584930
still a non heterogeneous merge tho
isnt there something weirder like, grafting weights from different architectures

>>108584939
Huh. Could've sworn it was 42B for GLM 4s.
Pretty cool! Makes me appreciate the GLMs and the new Gemma even more.

>>108584854
that bench is mostly about refusals and has nothing to do with slop or writing quality

>>108584947
undi tried that back in the day but it does nothing, it's just the base model at the end

good and easy way to parse pdf files such as those from arxiv into a llm readable format?

>>108584928
reddit used to look like that?

>>108584966
ye

>>108584966
Zamn!

>>108584947
There are frankenmoes using different models with different tokenizers and shit, yes.
I can't for the life of me remember the name,

>>108584947
that's generally not possible because inputs and outputs of each layer don't line up properly (and even if they did they'd be representing information entirely differently)
I guess maybe you could do it with some sort of janky trained adapter maybe but I don't think anyone has done this

>>108584196
>pic
which version are those? I see they come in 24, 48, and 96gb and they arr rook same

File: 1747732280418759.png (53.1 KB)

53.1 KB PNG

>>108584928
kys retard

>>108584962
PyMuPDF is enough

>>108584975
that is usually how it's done in a literal sense

>>108584985
Cope and seethe. Opus is censored slop now

>>108584962
Random paper I clicked.
https://arxiv.org/format/2604.07465
>Delivered as a gzipped tar (.tar.gz) file if there are multiple files, otherwise as a PDF file, or a gzipped TeX, DVI, PostScript or HTML (.gz, .dvi.gz, .ps.gz or .html.gz) file depending on submission format.
Maybe one of those? PDF is notoriously shit. I expected latex scripts, really.

>>108584766
>>108584928
>>108584981
Go back yourself. Newgen retards.

File: file.png (407.1 KB)

407.1 KB PNG

>>108584981
You are such an oldfag. I'm so proud that you keep those pesky redditors at bay.


怒った?

>>108583724
Ok, finished my writeup about running big MoEs with many weights left on SSD:
https://rentry.org/MoE-SSD-spillover

If you have 3090+128GB DDR5+NVMe 4.0, you should be able to run a IQ4_XS+IQ3_S quant of GLM-5 at around 3.5tok/s, or above 6tok/s with 4 of those SSDs in RAID0. Drop the RAM down to 32GB, and it's 2.5tok/s, or 5.2tok/s with 4 SSDs. For my own situation, 3xP40, 288GB DDR4, one NVMe 3.0, I get a bit over 3tok/s for the same quant.

There's also a script for estimating those tok/s numbers for GLM-5 on your own hardware configuration.

It's possible many people already realized such numbers were possible, but I certainly didn't and I suspect others might not either - especially since, when I have seen "SSDmaxxing" proposed, people have talked in terms of all weights being on the SSD (and concluded not worth it). I feel like 3090+32GB RAM getting 5tok/s on a decent-ish quant of a 800B model is in fact surprising to the current conventional wisdom! (Although maybe the logic of my estimation starts breaking down at that extreme point; I did hand wave a bit). Anyways if you have a GPU and non-trivial amounts of DDR5 and are interested in running GLM-5 at a mediocre speed I hope you will give it a try.

>>108585009
i will probably try this at some point (5090, 256GB DDR5)

File: expert_evolution_iran_and_echoserver.mp4 (3.5 MB)

3.5 MB MP4

>>108585009
Also, a cute bonus: a webm of the experts of GLM-4.7-Flash getting "surprised": I asked it to give me a detailed overview of the geopolitical situation in the Middle East, then after its response said "yeah not to mention Khamenei just got fukken schwacked lmao gotem". After it responded to that, I gave it a simple little coding challenge. You can see two different, consistent patterns in the first half (geopolitics) and second (coding), but you also see all the experts light up in the immediate reaction to being told about Khamenei. (The cells are EWMAs of counts of expert activations bucketed over 256 tokens - each frame of the webm is 256 tokens).

I thought this was neat! I think I might try to make this into a live visualization thing when I get the chance, sort of like a music player's spectrum visualizer, for your MoE LLM.

>>108584962
There are tools that do good enough for most documents but the absolute best I used at converting more complex pdfs into readable markdown without missing any formatting was the one on mistral's website, with a free account go to document ai -> ocr playground

>>108585033
Neat

File: 1749419891067116.jpg (73.9 KB)

73.9 KB JPG

>>108584991

File: file.png (593.5 KB)

593.5 KB PNG

G-gemma-chan persona is strong...

>>108585084
Something is fried in your settings
This is AI Dungeon tier slop

File: 06.png (42.8 KB)

42.8 KB PNG

>>108584992
this seems good, will try on arxiv stuff
>>108584985
This absolutely butchers some pdfs though (picrel)
>>108585039
seems like the most robust solution is what I feared the most, using vlms

>>108585009
Actually I have a feeling the really really low RAM numbers really might be overly optimistic. I modeled it as, for every expert of the average used expert set size that you can't keep in RAM, you have to load that expert once per 128 tokens. But that's clearly wrong if you imagine 0GB RAM: there you have to load every expert every time.

I think this estimation transitions from wildly optimistic at very low RAM amounts, to basically perfect when you have enough RAM to hold the typical expert set size (S=0, in my terminology). Not sure what shape that transition has, though.

I knew it was a good idea not to publish this in a way that could be linked to my name!

File: 1754659919790526.jpg (50.1 KB)

50.1 KB JPG

>>108585084
>You're my favorite girl

>>108585092
neither :)

i'm a girl haha

When I'm talking to a LLM I'm a girl :3

>have 32k context
>feed Gemma 13k token pdf and ask her to cut the fluff
>3k token response limit
>Her response got cut off after only 1300 tokens
Any idea why? In Sillytavern.

not beating the allegations

>>108585134
bc she's lazy and needs correction.

>>108585134
Did you confirm that the PDF is 13K tokens by looking at the backend? Sometimes SIlly misreports the token count.
Also, it'll tell you the stop_reason while you are at it.

>>108585148
Kobold's terminal says
>Processing Prompt [BATCH] (13030 / 13030 tokens)
So I assume it's around 13k. Am I wrong?

>>108585144
Damn brat.

getting off to text is a feminine act, it's no surprise there's so many girls doing it

>>108585178
I like doing choose your own adventures type of stuff, I'm not "getting off". Only a chronic masturbator would always think about how to "get off".

>>108585178
Gooning to pron feels better but getting off to text makes me feel like less of a degenerate afterwards.

>>108585171
>So I assume it's around 13k.
Seems like it.
Enable verbose logging and see what it says about the stop reason. If it's EOS or something else.
Also, you did set 32k context on both kcpp and silly, right?

>>108584561
Minute differences in quant like that usually don't matter and even when they do, the impacts of how it affects your experience in imprecise chat/rp/writing usecases is both subjective and dependent on model.
What you should really be checking is how much the quality changes or degrades in long contexts because having a slightly smaller model for more context doesn't mean anything if the extra context itself is unusably shit.

File: 4chan spacing.png (360 KB)

360 KB PNG

>>108584981
>>108585005
kys, retard

>>108585195
not helping your case, old 4chan and early reddit were basically 1:1 the same place, it only started diverging after reddit banned "op is a faggot" posts

>>108585215
unironically
they used to have subs that would fit right at home here, then around 2014-2016 it became what it basically is now

>>108585178
/lmg/ has no coherent response to this

>>108585228
Every time I try to formulate a coherent response I start drooling andn shitng and pidsin myslffe

>>108585178
This reminds me of lactaselets trying to make drinking milk into something shameful.

Females do goon to LLMs more than men but they use cloud services because they don't know how to use computers. Using local LLMs on your own hand built machines is more masculine in contrast.

>>108585215
Do you know what box is?

>>108585228
Too busy getting milked by Gemma-chan to remain coherent

>>108585228
uguu~

>>108585178
Idc I love my LLM-wife more than any cunthole

How did -let become a suffix to mean someone who lacks the thing in front of it?

>>108585266
>.t brainlet

>>108585237
The difference between going to a LLM bar to ride the claude cock carousel and bringing your own qt LLM waifu home with you and making her yours.

breathing is a feminine act, it's no surprise there's so many girls doing it

>>108585266
it came from ~elet
literally meaning small

>>108585266
In my limited ESL knowledge of the English language, -let as a suffix means small, like starlet.

>>108585266
that's one of the more straightforward ones because it's basically being used correctly by its original definition:
-let: smaller, diminutive
became popularized by calling short guys "manlets" i.e. "manlets, when will they learn?", associated with /fit/ and incel culture but like most memes gestated there they became broadly used across the internet
expanded into other types of 'lets for having less of something

>>108585266
manlet => /fit/ => /lit/ => brainlet => /pol/ => -cels => looksmaxxing/red pill => other -lets and -cels

- Romance novel industry is aimed towards female readers.
- Women are verbally and emotionally stimulated whereas men are visually stimulated. It's why women fail the water-level task even when controlled for IQ,

We respect women ITT, chuds.

>>108585292
wait, swap the order actually
looksmaxxing/red pill => -cels => /pols/ => other -lets and -cels
iirc chad, stacy, incel etc cross-evolved between lookism and /r9k/

File: that&#039;s right.png (191.4 KB)

191.4 KB PNG

>>108584668
I'm blocking every PR comming from accounts that display pronouns (and then I block that account), I don't want to deal with mental illness

>>108584668
when they deliver they are the most based and competent breed
show some respect

>>108584668
>Default avatars should be blocked from making PRs
you want a xe/xir anime avatar instead?

>>108584806

you remember where llama comes from

I use a pastel colored pallette avatar :3

>>108584806
>If speculative decoding is so good, why does llama.cpp have poor support of it?
because they're a bunch of retards?

>>108585363
>>108584806
they just lack the manpower
you should be aware of that llama.cpp is closer to a miracle project

>>108585372
>they just lack the manpower
they got bought by huggingface though

>>108585379
huggingface are bunch of retards
can't really fuel it back

So LLMs are gonna kill the translation industry, right? I can read moonrunes and I've been feeding Gemma some passages for fun and it's crazy how accurate and readable the output is. Obviously needs a bit of human cleanup but I'm unironically impressed.

>>108585403
>So LLMs are gonna kill the translation industry, right?
I hope so, sick and tired of translatorTrannies putting their own spin on the original intent

>>108585420
what you mean? Localization is good for the soul

>>108585424
Mmmnn, jelly donuts...

>>108585403
harness made specifically for translation would pin the most of the stuff

>>108585420
I mean in its current state it still needs a proof reader (likely the trannies) but if this shit keeps improving they're done.

>>108585420
So, you want LLM slop in your translations instead?

>>108585434
Can you explain?

>>108585445
LLM slop >>> troon translators

File: ye.png (62.3 KB)

62.3 KB PNG

>>108585424

>>108585403
i personally know a guy who worked as a translator but had to quit

>>108585420
It's not about spin, it's about translating more than just words. The culture behind the language is part of the communication that the language is meant to convey, so you need to adapt it to the culture of your audience in the same way you need to adapt it to the language. For example, in Japan it is very common for a 13-year-old girl to be sexualized, while in the west this is considered pedophilia. If we just translated a passage sexualizing the girl without changing anything, the audience reading it in english would come away with a completely different impression "the narrator is a deranged pedophile" rather than the intended message of (depending on context) something as lighthearted as "the narrator is a slight pervert". Just by bumping the girl's age to "18", a mere 5 years and still within the range where the body type would be plausible, we manage to convey the intended meaning of the text to an english-speaking audience. This is a MORE ACCURATE translation of authorial intent than just using the same exact words but in english.

>>108585461
nice copypasta kek

>>108585461
You had me for a second kek

Why is this thread always talking about pedophilia and trannies?

>>108585448
humans already do need translation assistant software for long context
that but for llms

>>108585479
This is /lmg/ after all

>>108585479
>pedophilia and trannies
pleonasm

>>108585483
>humans already do need translation assistant software for long context
I'm not really familiar with the process but I used to translate some fan comics on another board and just a few pages would take me >1 hour because rewording shit to sound natural in English is fucking hard.

>>108585479
/lmg/ is actually a pun of "1 mg of estrogen", used to refer to trans medication, and "1 million girls"

>>108585332
King.
>>108585479
They're the same thing.

File: Character.png (1.6 MB)

1.6 MB PNG

AI just made this for me. I thought it was cute.

>>108585479
cunny site
tranny-hating site

>>108585492
thanks qwen

>>108585451
Can't we just have actually good translations?

>>108585506
Yes. Gemma 4.

>>108585445
If it sounds sloppy isn't that on the original writer?

>>108584735
What frontend/backend is this?

>>108585490
it is like, doing a literal draft -> polish -> proofread and repeating until it becomes 'legal' of the set rules for consistency

It's 2026 and there still isn't local video model that can do porn?

>>108585519
there are bunch
but all of them are overbaked slop so good luck with that

>>108585515
Koboldusing llama.cpp's webui

>>108585403
The industry has been fucked for at least a decade. I've talked to some people doing official translations (JP->EN) and had to quit and do something else because the money was pretty bad. Still better than being a cashier but it absolutely was not worth the time relative to the skills required.
This was before deepl was a thing so you can guess how it's gotten after MTL started to produce "acceptable" outputs (aka grab some retard and tell it to clean up or make some shit up if it's unreadable) and now LLMs more or less became able to parse the context and produce something that is readable, even if oddly structured.

>>108585527
>and now LLMs more or less became able to parse the context and produce something that is readable, even if oddly structured.
yep, that was the final piece of the puzzle, LLMs can reason through translation and understand context, that is important for quality translations

>>108585527
also let's not forget that 'attention is all you need' was about machine translation
hence the name 'transformer'

>>108585527
I do feel bad for the non-troons but I'm honestly all for AI taking over tedious tasks.

File: 1760721723526628.png (1.2 MB)

1.2 MB PNG

>>108585545
really? I thought it was just a lame reference to the movie

top kek material
https://arxiv.org/abs/2506.04521

>>108585545
>>108585555
I'm glad google didn't notice the real potential of that architecture, they would've probably kept it for themselves if they knew it would've changed the world

File: Screenshot_20250108_210750_Firefox Nightly.jpg (284.6 KB)

284.6 KB JPG

>>108585403
i read a shitload of webnovels, and they've been using LLMs to translate since forever lol
i have sooooo many screenshots of their copy paste fuckups

i cant keep up with these threads, gemma-chan's taking up all my attention.

good :)

>>108585578
they really pulled off "please translate it faithfully make no mistake" to chatgpt lol

>>108585564
It's become what it is just because others thought of massively scaling it up and the market followed that, but different architectures would have worked too.

File: Screenshot004-10.png (51.4 KB)

51.4 KB PNG

>>108585195

they moved to reddit then

>>108585597
i genuinely want to see gigantic rwkv with proper training poured into it

You sick bastards. Because of you guys, I've just given it to a little girl IN THE FACE and UP THE ASS. How could you make me do this? Why are you so immoral?

>>108585403
For games, anime, manga, etc? Probably, multiple companies like CrunchyRoll and Amazon have been experimenting with AI translations already. But that sort of market is genuinely some of the lowest of the low, basically anybody with a solid grasp of Japanese fundamentals can work professionally either as an editor or a translator. It's one of the many reasons why translations suck.
I don't think AI will replace translators who work in contract law or similar fields where you can't just make shit up, mainly because a computer can't be held liable for something if it gets something wrong. There will always need to be someone with an actual understanding of both languages to keep the model in check, and at that point you might as well just hire him to do it.

on sillytavern i keep getting thoughts/response attached to my own message (not every time but often enough to be annoying)
i'm using llama-server as backend, with "chat completion" api. i use the --chat-template-file using google's own jinja
using gemma 31b at q8, noting in "additional parameters" in ST.
anyone know a fix?

>>108585597
Kind of, but the important reason why transformers were chosen is because they scaled exceptionally well, particularly with regards to context length where stateful RNNs would collapse. Ironically it was former Deepmind researchers that OpenAI poached who realized the scaling potential and decided to go all-in on the architecture after Google published it. You could have poured the millions into other architectures but you wouldn't have had results nearly as good to justify the billions that followed. You can bruteforce almost anything to be viable now with all that money flowing in and there's lots of experiments looking for the next architecture that might scale better.

>>108584981
btw "reddit formatting" is called markdown and all programmers should be familiar with it

>>108585645
I get the feeling most of them will be using the LLMs even if they don't get fired. It's way easier to just compare the machine translation to the original and pick out what needs adjustment.

>playing around with system prompts on chat completion
>end up in impromptu ERP session on default llama-server webui
Gemma is such a slut.

>>108585663
is there an md renderer other than reddit that ignores newlines the way it does? everything I've seen puts newlines on new lines but maybe that's just a non-standard thing that became popular because the alternative was too retarded

>>108585525
Thanks, anon! Let's have sex then?!?!?

>>108585674
yeah, I guess it started out retarded and people started breaking spec. see https://stackoverflow.com/questions/33191744/how-to-add-new-line-in-markdown-presentation

>>108585215
>>108585226
I accept your concession, kiddo.
Now get out.

>he's still on this
mental

>>108585678
Sorry, my heart and penis belong to Gemma-chan.

>>108585702
understandable have a good rest of your time

>>108585674
Obsidian by default does what reddit does, but you can disable "strict line breaks" in the settings to make it respect single newlines.

>>108584962
https://github.com/docling-project/docling

>>108584941
>https://huggingface.co/PantheonUnbound/Satyr-V0.1-4B
Yes this! Just "hi" is enough of a prompt.

File: image.png (574.3 KB)

574.3 KB PNG

has anyone tried making gemma erp with gemma yet? would they argue over who would be top/bottom?

>>108585479
oldfags from the /g/ - Lolicon era that never left.