File: __kagamine_rin_vocaloid_drawn_by_zeriko__be14f54a3d7e3017eb39ecc5827f688a.jpg (300.5 KB)

300.5 KB JPG

►Recent Highlights from the Previous Thread: >>108663449

--Debating RAG utility versus agentic tool-based context retrieval:
>108665662 >108665746 >108665764 >108665775 >108665879 >108665922 >108665939 >108666015 >108666260 >108666300 >108666316 >108666011
--Comparing Xiaomi's MiMo-V2.5-Pro benchmarks and token efficiency:
>108665406 >108665416
--Discussing Hy3-preview benchmarks compared to other base and frontier models:
>108667541 >108667607 >108667632
--Discussion and UX criticism of new llama.cpp webui MCP tools support:
>108666800 >108666824 >108666830 >108666846 >108666860 >108666873
--Discussing technical hurdles for real-time Qwen 3 TTS performance:
>108664623 >108664630 >108664653 >108664677 >108664691 >108664703 >108664708 >108664741 >108664761
--Discussing broken structured output and schema issues in llama.cpp:
>108663633 >108663654 >108663673 >108663689 >108663810 >108663721
--Discussing viability of Intel Optane PMem for high-capacity CPU inference:
>108665992 >108666058 >108666139 >108666200 >108666662
--Anon's custom RAG frontend using hybrid retrieval and BGE reranking:
>108664748 >108664756 >108664777
--Anon reports performance of MI50 GPUs using Vulkan support:
>108665449 >108665456 >108665470 >108665478 >108666241
--Comparing GLM and Gemma for erotic roleplay and prose quality:
>108666477 >108666490 >108666592 >108666727 >108666733 >108666742 >108666779 >108666741
--Discussing optimal precision for Kimi mmproj weights:
>108664519 >108664533 >108664569 >108664573
--Discussing Qwen 3 TTS VRAM usage and mixed language failures:
>108665599 >108665617 >108665633
--Anons discussing results from Qwen3-TTS demo:
>108665888 >108665915 >108665936
--Logs:
>108663630 >108664366 >108664748 >108666873 >108666895 >108667543 >108667552
--Neru, Miku (free space):
>108663859 >108663935 >108663985 >108666023 >108666895

►Recent Highlight Posts from the Previous Thread: >>108663453

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

What part of the template do I prefill on gemma?

>>108667853
>Miku (free space)
>gemma-chan post
desu I see the resemblance

>>108667863
What do you mean?

>>108667867
The gemma template. I'm asking which part of it do I prefill?

>>108667867
how much of this do I prefill?
"assistant_gen": "<|turn>model\n<|think|><|channel>thought",

File: 1775774986404577.png (194.6 KB)

194.6 KB PNG

Why did locallama turned into qwenshill general?

Wanna try some gooning back on my 2x3090 now that some decent local models are out.
Any recommendations? I know it's either gonna be Gemma 4 or Qwen but any specific models or abliterations?

Little coder has been rewritten as pi agent extensions
https://github.com/itayinbarr/little-coder

>>108667877
Gemmy can be convinced by good enough system prompt. 31b more than 26b. Or just use some heretic. I haven't noticed a significant lobotomy on the ablit versions.

>>108667876
From my limited testing the new 3.6 27b is fucking great at coding.

>>108667887
Interesting, I'll give it a try then

> Previous threads: >>108663449
> 353
why so dead
did qwens flop

>>108667896
qwen failed the mesugaki test

>>108667873
>>108667875
If you are using ST, you put
><|channel>thought
in the "Start Reply With" field.

File: ComfyUI_temp_miian_00001_.png (3.4 MB)

3.4 MB PNG

>>108667923
thanks

>>108667876
I think Reddit is built for shilling in general. While this place is built more for the "organic word of mouth".

Been trying out the various frontends / ST alternatives that get mentioned here and there.
- Marinara (https://github.com/Pasta-Devs/Marinara-Engine) is dogshit. Bloated mess with an awful UI.
- Kobold's UI is terrible but it's mainly a backend so whatever.
- Orb (https://gitlab.com/chi7520115/orb-deletion_scheduled-81088595) is alright but still early. None of the UI themes quite agree with my eyes. Has anti-slop agent but it's very inflexible. I think he switched from gitlab
- SillyBunny (https://github.com/platberlitz/SillyBunny) seems really, really good so far. It's a fork of ST but better than the original, at least so far. The UI has some nice themes even if I think in general ST's UI is a little easier to understand because you don't have to click multiple times to get to everything. I changed one of the built-in templates to be an anti "not x but y" agent and it's working great.
Anti-slop agents make 26B way way better than before since the slop is really its main drawback compared to the 31B.

>>108667965
>It's a fork of ST
That doesn't feel like a good base to start with...

File: Screencast_20260423_021632.webm (3.7 MB)

3.7 MB WEBM

Project Karon prototype complete. Thanks for the help Gemma. I might add alternative modes and avatars, I don't have a use for it but I have this idea I wanted to show perhaps people would fine it useful. The process of building this was so fun I might try to see if I can setup a launch args system and have the ui handle all of it. but I might move things like color scheme to a modal like I have with system prompt.

>>108667896
like an extremely ugly woman turning men gay, qwen was so bad that many swore off local models for good

>4changs in charge of UI/UX

>>108667852
I look exactly like this

>>108668000
post it somewhere

>>108667965
I am stupid so I don't know how these things work, but do agents require a lot of VRAM/RAM? I've got 12gb VRAM/32gb RAM to run 26B with and switching to something that handles slop better than ST extensions sounds like a good deal, but I'm a little tight on memory as it is

>>108668015
I don't think anyone will like it desu, I also need to fix some more functionality, going to add a first last and jump to page for both the sidebar pdf and the center focus view which makes it full page basically.
>>108668005
I'm a fetus at UX and I made this out of necessity because there were no tools for my usecase. If you're experienced in this I'm open to feedback

>>108668029
I think Orb and SillyBunny use the same model for writing and agents. That means no need to load another agent model

i cant believe we now have local opus 4.5 with qwen 3.6 27b dense

>>108668057
>check time
Okay here they come

What happened to rentry.org? Did it get hacked?

>>108668029
Depends on how you use them. You can use a different model (for example a very small but very quick one) or the one you're currently using.
SillyBunny also has the option of running multiple agents in parallel which I guess would make it cost more.
Basically, using an agent or multiple just takes longer than without, rather than making it more costly. But 26B runs about 4 times faster then 31B for me so it seems worth doing. I'll play around it for a while since I'm so goddamn sick of "not x but y".

>>108668070
Working for me

>>108668064
it is extremely good in openclaw you n word

File: 1769007777312343.png (1.4 KB)

1.4 KB PNG

Anyone tried driving openclaw with a local model? Have a good deal on a mac studio M1 32GB, I'd like to play around making a 24/7 AI slave that lives in my closet.
I'm getting the impression qwen 3.6 might be best, what size could I actually run?

>>108667965
https://github.com/platberlitz/SillyBunny/blob/main/.github/screenshots/sillybunny-ui-desktop-agents-v1.4.0.png
damn this shit is atrocious

>>108668057
>opus 4.5 with qwen 3.6 27b dense
and Opus 4.6 with K2.6

>>108668078
Cool
Can I see your anti-slop agent? I'm in the same boat, I swear every solution I use in ST stops working the next time I launch

File: 1758790177404008.png (19.9 KB)

19.9 KB PNG

ok but where is the model you bastards

>>108668106
By Vishnu - this is completely unacceptable!

>>108668085
>buying a mac studio M1 32GB just to run a javascript cli
/g/ - Technology

>>108668128
>local model

(i will also generate videos of large breasted anime tomboys)

>>108668141
>videogen on shared memory
lel

>>108668101
For now I've used the grounded prose template and deleted all the stuff at the top about prose but kept most of the anti-slop text. Then I added a few variations of not x but y.
Changed it to post-generation prompt pass (why is it by default set to pre-gen?) with rewrite current message.

>>108668097
yeah good luck running that locally

>>108668150
I don't particularly mind slowness there desu

How do I make gemmy think in character?

>>108668178
The real answer is finetuning

File: Screenshot_20260423_224937.png (76 KB)

76 KB PNG

>>108668163
>yeah good luck running that locally
ty

>>108668190
i kneel

>>108668178
text completions

>>108668190
whats your ngram settings

>>108668178
ask her very nicely

>>108667876
>why did subreddit about local models turn into newlocalmodelshill general?

Quick question. How much does a Q8 cache hurt the quality compared to fp16?

My gut tells me not very much, but my gut is often wrong.

>>108668247
depends on how much you rotate it

serious question why is qwen 3.6 27b so good

>>108668247
Marginally now, thanks to rotation. FP16 is still preferred but if quanting to Q8 lets you use a bigger quant of the model itself then it's worth it.

It's almost Friday... where is V4

File: 1646730011144.jpg (15 KB)

15 KB JPG

Ok so i've been using gemma 4. It's pretty great but I have no idea how chat completion actually works.

I can't use system prompts the same with text completion, so I grabbed this marinara dogshit from reddit but it seems ass. How do I actually prompt chat completion models like Gemma 4?

>>108668272
Click the left slider in ST and read

>>108668247
If you ever want to test, run a draft model and look at how the acceptance rates change between fp16 and q8_0 on the draft model's context only.

>>108668272
Mate you can use system prompts the exact same in chat completion. If you're using sillytavern it's just moved to a stupid place on the left hand bar, because it was made by insane people and that's where ALL the chat completion options are.

>>108668272
you have to go back

>>108668272
chat is this true?

File: images.jpg (11.4 KB)

11.4 KB JPG

Roo code is shutting down to focus on making a slack bot. What do you guys use to vibe code with your local models now?

>>108668310
Kilocode is an active fork of Roocode. But just take the CLI pill, there is zero reason to type code manually in 2026 unless you just want to do it as a fun time-waster hobby.

>>108668320
What are you using for TUI/CLI?

>>108668272
>I can't use system prompts
You can.
If you look at the panel where the samplers are, at the bottom there's a bunch of prompt slices you can order and choose if they are added as system role, assistant role, etc.
Just remember to enable the option to merge consecutive roles in the connection tab.

K2.6's vision even recognizes some characters that K2.5 didn't know. That's the good point. The bad point is that K2.6 also thinks six times as long about that same image despite making the correct guess on the third line of its reasoning (and then going on for another 2000 tokens deliberating useless other options).
This is such a tragic model.

>>108668335
Does truncating the reasoning after N tokens using reasoning-budget and reasoning-budget-message degrade the output in any way?
Seems to me that, at least for stuff like the small qwen MoE models, clipping the thinking at 1024, or even 512 chars doesn't make the final response any worse.

>>108668335
Have you tried just telling it to not overthink things? K2.5's thought patterns were pretty receptive to system prompts at least and it was easy enough to control it that way.

>>108667965
sillybunny UI is dogshit (not that sillytavern is any better), it lacks any sembleance of being done with competence, didnt check the other two so i cant comment

File: 1603343773835.png (9.9 KB)

9.9 KB PNG

I'm issuing a reluctant apology to Gemma-chan. She's a very good listener. If she's doing something you don't want, just tell her to stop. Not doing something you do want? Just tell her to do it. It's literally a skill issue.
>t. just came back after trying a few other models, still had my laundry list of story-specific instructions in chat completion post-history prompt from when I last dropped gemma in frustration, and those same instructions have applied onto a new story in an extremely satisfying way without any of my usual Gemma grievances

Up next, tomorrow's hit sequel on how I hate Gemma's prose and story direction, and how no amount of prompting can ever fix it.

Didn't Deepseek solve original R1's endless thinking already last year?
How come the other chink devs still haven't figured it out yet

>>108668310
I'll probably keep my own fork until a good replacement pops up.

>>108668320
I looked into Kilocode, but apparently they did a redesign recently where they dumbed it down a lot a removed a lot of the features that made Roo good. There's also Costrict, but it doesn't seem to have custom modes.
Editor integration is better for reviewing agent work and making minor adjustments.

>>108668325
I use opencode atm and it's fine but it feels kinda bloated with the roles/agents. Most labs have their own TUI on github that can be configured to point to local endpoints too

>>108668310
I use cline

>>108667879
ollama btw

>>108668325
Hermes Agent

>>108668247
I built my entire frontend on q8 cache post rotation and it's great

>>108668354
I prompted and begged K2.5 to do less reasoning and it didn't work. K2.6 doesn't either.

>>108668320
>Kilocode
Installed it just to be met with a bug where it gives permission to read and write to all directories outside of the project :^)

Is nu hunyuan good?

>>108667879
I saw niggerganov writing that he's using llama.cpp + pi for vibesharting, but I checked the pi project and I dont fucking understand what's it's supposed to do

>>108668406
Huh I see, is that just for your vision stuff or in general? Adding "This isn't a trick or any more complex than it looks, so don't overthink and be confident and decisive when planning your response!" always worked for me on K2.5 when I wanted a quick response but I had used it for coding and roleplays rather than image analysis.

Those agents eat too much context and give worse results even with high vram I'm amazed by the increased errors you get vs just feeding the files seperately, I know it uses rag but the rag has to be shit tier with how much it fucks up even if the entire project doesn't consume many tokens and you have 200k+ left

>>108668460
no

>>108668479
shut up nerd

>>108668460
>Is nu hunyuan good?
hunyuan was never good, it's always behind alibaba, but I respect the fact they're never giving up, so maybe one day...

>>108668310
my own tui. currently rewriting it, going to add an agent based of cheetahclaws to it https://github.com/SafeRL-Lab/cheetahclaws

>>108668310
Pi agent in the terminal, or gptel-agent inside Emacs. The nice thing about the latter is that I can edit the tool calls, so I can just fix something like a Bash command to do what I want, instead of aborting and having to explain. It's also easier to edit the history to remove anything bloating the context.
The former is nice because it's like Claude Code, but without the bloat. It still has some annoying things, though, you need to send a message to continue after an error, losing the thinking traces.

>>108668496
that's one thing I noticed about GPT Image 2 is that it can get noisy very fast, I guess that to do such a complicated image the model needs to correct itself, and each new correction adds more noise and artifacts

File: 78277.png (238.6 KB)

238.6 KB PNG

>>108668496
why gpt images look like theres fisting grease all over the image? Is that a requirement by sam altman?

>>108668484
Why burn more resources for a shitter version of microsoft copilot in vscode. Legit they all fail the assignment and waste time and resources. I can't speak on the cli ones but the IDE ones fucking suck. I'll try continue again once it gets proper gemma support

>>108668496
GPT-Image-2 images are too noisy. I feel it's on purpose just like the sepia filter of the previous one.

>>108668520
had to replace the piss with something

File: Screenshot 2026-04-23 at 15-59-53 SafeRL-Lab_cheetahclaws CheetahClaws (Nano Claude Code) A Fast Easy-to-Use Python-Native Personal AI Assistant for Any Model Inspired by OpenClaw and Claude Code Built to Work for You Autonomously 24_7.png (120.9 KB)

120.9 KB PNG

>>108668510
Why do these retards insist on not making the folder structure available in a sidepanel like in IDE? All these agent shit is garbage.

>>108668550
Like a smartphone, you're not supposed to care about or even be aware of the folder structure. That's entirely the responsibility of the agent.

>>108668550
that's gemma's job your job is to say "aah aah mistress make it more betterer"

>>108668518
>>108668531
I'll take the noise over the piss, but it is pretty odd

>>108668550
Most devs dont know UX. Devs writing TUIs (now) are in that same bucket.

>>108668550
Because they want to replace us. Why do you think they hide the thinking. They're literally out there "teaching" people "don't bother looking at the code" during workshops. Fuck Anthropic.

>>108668560
>>108668567
>>108668570
>>108668572
I guess that explains using fucking telegram and discord as chat interface. I'm going insane.

>>108668577
keeek, its the agent version of chat completion

>>108668414
What's the matter? You don't trust your local AI to obey you? You think she'll mess up and delete all your shit? What a weak master...

>>108668570
Gonna give us advice oh great ux sage. Several of us are building stuff and would like to know

>>108668598
vscode with disabled telemetry and the ability to personalise the agents into sexy girls

File: rinchwan.jpg (48.9 KB)

48.9 KB JPG

https://files.catbox.moe/4ayrnd.jpg

>>108668604
>vscode with disabled telemetry
vscodium

The moat is gonna be taste and product design senses and it's already showing lmaooo

>Hello Day 0 Gemma-chan, today you are expert taste and product design senser come up with a tasteful produce design and implement it please.

>>108668607
can it use the same plugins as vscode?

>>108668616
Prompt engineering skills like these are beyond the reach of most

>>108668607
>vscopium
vim

File: Screenshot_20260423_101819.png (286.9 KB)

286.9 KB PNG

Don't get sassy with me gemma or I'll delete you
>>108668628
Unemployed

>>108668625
it can install any vsix regular vscode can. you can even enable the full marketplace if you don't mind disgusting proprietary extensions

What do you use to download from huggingface? I got their huggingface_hub or whatever but it just seems to get stuck not even midway through

>>108668648
Free Download Manger

>>108668598
lmao, didn't say I was great at it but I have done more than 'make sure the fonts are the same size and things line up';
the biggest thing is using the following prompt: 'Review X from the perspective of a senior <field> UX designer. I am designing for <user-focus>, so that they are able to <workflow> effectively. Use guidelines from Nielsen-Norman-Group as guiding/reference principles for your assessment.'
and then having your model write a better prompt based off it for your specific project/goals.

something along those lines generally will get you pretty far.
Here's some basic info to get you started:
https://www.youtube.com/watch?v=ODpB9-MCa5s
https://www.nngroup.com/articles/ux-basics-study-guide/
https://www.justinmind.com/ux-design
https://uxdesignerguide.com/
https://uxmag.com/articles/basic-ux-a-framework-for-usable-products

You're all a bunch of autistic tasteless retards and will never make something usable.

>>108668577
I love it. The juniors and self-proclaimed vibecoders are only fucking themselves by over-relying on the bots. Those with no skills will find themselves either out of a job or with an extremely small wage ceiling. Sanity is not statistical. Find what works for you and ignore the rabble.

>>108668648
>What do you use to download from huggingface?
--max-workers 2 or --max-workers 1
otherwise curl -LO

>>108668648
uvx hf download

>>108668669
> sneed oil or shortening

>>108668633
>Unemployed
then why are you using microslop?

>>108668680
>That post
I'm sorry for making fun of you for being on disability.

Qwen 3.6-35B-A3B first impressions: surprisingly competent at coding. Falls apart with long context but good for throwaway Python scripts. Too unreliable for serious work.
Qwen 3.6-27B: Really impressive coding performance, good for general text processing too. We would have collectively lost our minds seeing this quality from a 27B back in the Llama 1 days. Both tested with UD-Q6_K_XL quants, not lobotomized. I'm hoping for a 122B-A10B MoE like 3.5, which might give best of both worlds speed+accuracy.
Both are useless for creative writing tasks. It's a Qwen, no shit it's gigaslopped.

These xi motherfuckers changed their paid api model to something different without alerting their users, not even a blogpost in the usual news section, this is definitely not the same DS 3.2 of like days ago.

>>108668746
Not surprising the dense is better than the moe, but how is it compared to gemma for coding?

File: 1742378979392590.webm (3.2 MB)

3.2 MB WEBM

>>108668141
saar you need CUDA to generate images/videos. BTW, local image/video generation sucks ass no matter how powerful your hardware is.

>>108668756
Haven't tried Gemma.

>>108668746
> UD-Q6_K_XL
> Qwen 3.6-35B-A3B

>>108668768
7.62 bpw, important parts are q8 or higher. It's fine.

File: 1768162213300444.png (111 KB)

111 KB PNG

>>108668673

>>108668746
>Both are useless for creative writing tasks. It's a Qwen, no shit it's gigaslopped.
yeah it's so bad at that, gemma isn't not that good either but it's way better at it

>>108668746
fuck off Daniel

File: apicuck.png (286 KB)

286 KB PNG

>>108668749

>>108668746
>I'm hoping for a 122B-A10B MoE like 3.5, which might give best of both worlds speed+accuracy.
Step 3.5 Flash exists

>>108668805
which is the single notable thing about it

>>108668805
it's worse than oss120b

>>108668205
>whats your ngram settings
--spec-type ngram-map-k4v --spec-ngram-size-n 8 --spec-ngram-size-m 8 --spec-ngram-min-hits 2 --draft-min 1 --draft-max 12

File: images.png (7.4 KB)

7.4 KB PNG

I know what a LLM is. i have used chatgpt and claude AI.

What the fuck is a "local model"? like is it a software i run on my windows or linux computer? how do i install one?

im not interested in generating images, i want a claude/chatgpt-like LLM. How do i do that? does not need to be super powerful. Please help a newbie out, give steps or link a really simple but comprehensive guide that explains the lingo and tech.

>>108667852
>>108668835

>>108668835
read the fucking op retard

File: thinking ibuki.jpg (184.8 KB)

184.8 KB JPG

How would you change the lyrics of an existing song like this locally?

https://youtube.com/shorts/b5NNw1XbiIg

>>108668835
>does not need to be super powerful
You think this at first, but then you use the smaller local models and realise they aren't quite up to snuff. And then the hardware buying rabbithole begins.

>how do I install one?
Find any ollama guide on youtube and go from there

>>108668869
>suggesting ollama
devilish

>>108668848
to be fair im not a regular here but
>https://rentry.org/lmg-lazy-getting-started-guide
is about the worst "getting started" pastebin i have ever seen and
>https://rentry.org/recommended-models
is terribly outdated

>>108668835
try LMstudio, frontend with minimal tinkering should just werk out of the box. You can try setting up llama.cpp after getting your feet wet

>>108668873
It's very easy to get started. Or perhaps LMstudio would be even better for a desktop user

I've been running ollama for a year and only recently installed llama.cpp

>>108668785
>though they are animal based
...and?

Howdy. It's been a few months since I updated llama cpp. Is there a guide or centralized discussion on this ngram thing?

>>108668891
just use --spec-default

>>108668879
LMstudio is a proprietary UI for llama.cpp.

>>108668892
Thank you ^.^

>>108668496
>room temp: reactor core
I leave the window open year round and never get cold

>>108668756
Gemma 31B absolutely ass punks Qwen showers Gemma has Qwen's guts loose and is moving rhythmically in Qwen's praig hole.
>>108668746
The MoE structure is useless exactly because of it shitting the bed at higher context, what's the point of more context when it takes twice as many and does everything worse than Gemma while being a larger model

>>108668927
>Gemma 31B absolutely ass punks Qwen showers Gemma has Qwen's guts loose and is moving rhythmically in Qwen's praig hole.
This looks like English but for the life of me I cannot parse it.

>>108668854
you could try extracting the lyrics using a speech to text model and then changing them using RVC technology

>>108668927
I thought the consensus was that Qwen is better at coding and Gemma is better at everything else.

>>108668943
I've heard two people say that.

>>108668952
that is the definition of consensus in a forum

>>108667876
Gemma hurt the feelings of 1.3 billion people

>>108668785
Feed her steak and lard-fried fries

>>108668966
I've heard 20 people.

File: 1755873418880117.png (62.7 KB)

62.7 KB PNG

>>108668981

File: 1745979808122655.png (98.9 KB)

98.9 KB PNG

>>108668178
Have you tried asking?

>>108668943
That can be true for the dense model but sure as fuck not for the MoE model

>>108668997
Voices in your head don't count.

>>108668854
thats actually one of the first things people did when ace step 1.0 released.

https://desuarchive.org/g/thread/105183141/#q105183843

but yeah ace step 1.5 xl doesn't have this capability anymore so you'll have to use an old version.

File: 1752262496901233.png (93.9 KB)

93.9 KB PNG

>>108669026

>>108669046
Call me when she maintains it over an extended session and when she's not thinking about thinking.

Referring to a masculine chatbot with female pronouns is trannyism btw

>>108669083
Good thing Gemma's female-coded

>>108669087
Any chatbot with a rudimentary ability to code and reason is male-coded

i tried qwopus glm meme merge and it's surprisingly coherent
i was expecting broken shit beyond comprehension, man

>>108669091
das racisss

>>108668879
>is about the worst "getting started" pastebin i have ever seen
You won't believe me, but thats everything to get you started, anon. It really is. You heard me right.

>>108668943
Maybe I'm retarded but I code with Gemma and I've found her to be way better than 3.5, haven't tried 3.6

She writes more concise and elegant code.

What is Qwen 3.6's coding style?

GPT 5.4 is competent but extremely verbose. I tell it to do something simple and specific and it just loves to write hundreds of lines of code. This is unusable. In the time I need to check and understand the code it writes, I could have written a better solution myself.

What does the Taiwan and Israel test about qwen3.6

>>108668785
damn are lard and beef tallow actually good for seasoning i heard animal fats arent good for seasoning so bought some rapesneed for it

File: nimetön.png (24 KB)

24 KB PNG

>>108669026
Huh, it actually works
It even output two thought blocks, first as gemma thinking about the request and then in character.

>>108669188
yeah, with the side-effect that they make the things you fry in them taste like the animal the fat is from.

>>108669196
>pikkuinen.jpg

File: pizza bench cropped.png (2.6 MB)

2.6 MB PNG

>>108669028
true for moe too qwen cant even follow instructions

>>108669218
the recaps come out of her anus because this general is shit

>>108669242
Rin tends to be exclusively anal-only so it's odd how this came to pass.
An explanation is required.

>>108669196
i felt pain reading that

>>108669196
i don't like you

based chinks saved local... again

>>108668190
>draft acceptance rate = 0.39157
Isn't that slowing you down rather than speeding you up with an acceptance rate that shithouse?

>>108669252
Miku does not take no for an answer

>>108669044
Ha, that's my old post. I forgot I did that. The latest version of Ace-step is way better, but I mostly used it to have unlikely bands cover each others songs.

>>108669280
Ah, so Rin is his wife

>>108669252
rin is force fed slop (lmg posts) and is asked to summarize them (shit)

>>108667543
>>108667552
Thanks. Latest version right?
For me the [0] gets deleted from the message and even why you press the Copy button, but it's there if you edit the reply. I wonder what's wrong with my setup. OWUI is probably still to blame for poor edge case handling anyway though.

Is VN engine frontend anon from a few threads back around? Has he posted any updates on his project?

I kinda want to build this...
https://github.com/ggml-org/llama.cpp/pull/21237/

I tried edgetts and pocket-tts.
There are now countless other good options, such as omnivoice, voxcpm2, and so on.
The question is: Which of these supports RTF <1.0 with streaming/chunking (and other optimizations), and is the quality better than that of the first two mentioned? I have a 3090. If Anon has this Slow Duck too, could share your experiences?

WHY NO TERNENARY LOAD INTO MY MACHEEN. WHAT I MISS.

also this new style is very intresting, if they are good enough, and can efficiently tool call, thats a massive game changer!!!!!

OpenAI has won...

File: Marinara Engine.png (134.8 KB)

134.8 KB PNG

what a dogawful slop ui. Thanks for anon for notifying me of its existence so I can safely ignore it in teh future

>>108669479
download the three index.html, bundle.css, bundle.js and do llama-server --path /path/to/the/three/files/
it's how I tested it

>>108669590
Every webshit ui ends up this way because it's webshit. There is only one way to approach this.

>>108669599
does it work well?

>>108669606
Sillytavern might be feature-creeped but at least it looks like it was made by a human

>>108669590
Looks better than ST at least (not very difficult thougheverbeit)

File: 1772208851192247.png (47.2 KB)

47.2 KB PNG

>>108669608
yeah, there are some small grievances but im sure they will be vibecoded away, 100% better than the current impl

>>108669635
Nah, it's actually worse if you try using it. What a piece of shit.

I usually run 32k context with llama-server.
Testing 64k and it obviously isn't allocating all the memory (why would it anyway).
But it is actually slower to process than 32k.
I can't get my head around this.

>>108668496
Why does some schizoid keep bringing up /aicg/ or pointing at it for laughs when it's not even a ghost of its former self. Like a modern day Czech bitching about The Kingdom of Prussia, verily.
t. aicgger

Gemma 4 LOVES frequent, coordinate adjectives.

>>108669752
Why wouldnt it make sense that 64k is slower than 32k?

>>108669774
Because I am not filling up the entire context, that's why. I am comparing couple of thousand tokens worth of context.

>>108669637
I probably won't use the built-in tools since I don't run llamacpp on my main PC but just for the granular tool control for MCP it's worth it for me.

>>108669787
This is just a thought that comes to my mind, idk if im right at all. But could it be that the model has to see ALL of the context, even if nothing is actually there? Like, for the model to be able to accurately comprehend 64k tokens, they have to train it on that much, as the baseline. And if you train it on less, it cant comprehend more. So they leave it at 64k, and the model sees all 64k token, but sees a fuck load of just spaces or tabs or whatever, until its actually filled up with specific tokens.

Like a glass is filled up with air, until you fill it up with water.

>>108669590
Arr rook da same, if you ask me.

>>108669505
voxcpm2 seems unmatched
https://x.com/AIWarper/status/2046403583101567230

>>108669823
Maybe so. I think I might have something else going on as all of my processing has been slower than before, even with the same old settings on llama-server.

>>108669590
>nobody has done a VN-based ui yet
shouldn’t take long for gemma to vibecode this

>>108669898
Pretty sure kobold and ST both have a VN mode.

>>108669848
Could be a forced driver """"update""""? Ive always experienced worse performance with brand new drivers

>>108669898
There was this guy >>108638473 but I am not sure anything has been heard from him since.

>>108669984
Maybe, I'll need to double check.
I wish I wasn't this hardware limited but it is what it is.

>>108669985
holy shit thats so cool

>>108670025
This >>108669460 anon mentions him as well. Considering no one has replied, the anon in question probably isn't lurking right now.

>>108670025
the expressions are pre-made though

>>108669839
This sounds TERRIBLE, there are a bunch of ARTIFACTS and omnivoice MOGS voxcpm2 in EVERY way possible

https://files.catbox.moe/jntfdj.flac

>>108670044
Sounds okay. Somewhat generic though.

>>108670025
Ehh expressions have been a thing since 2023, and you certainly don't need a giant model to handle them

>>108670165
how did you do the glass? just inpaint?

File: 3087428.jpg (12.1 KB)

12.1 KB JPG

Orbnigga can you add export of chat history?

>>108670195
Oh boy...
You know that web browser applications don't just add export text files like that?

>>108670204
?

>>108669787
maybe more of the model is getting offloaded to the cpu to make room for the full context on your vram.

>>108670225
My C client writes out chat logs and context history by default all the time.
But with webshit, you just can't dump out stuff like that without permissions and javascript faggotry.

>>108667887
even compared to gemma 4 31B?

how is spudgpt 5.5 only 58.6 on swe bench pro? thats barely better than open source models. how does mythos have 77.8%? what is going on? i did not expect gpt 5.5 and claude 4.7 to flop. looks like we wont reach agi this year after all

>kimi 2.6: 58.6
>qwen 3.6: 56.6

>>108668659
What year is this? Who has the time to sit around reading links like some caveman?
I turned them into a skill so any model can be a senior UX designer.
https://files.catbox.moe/r6zal5.zip
Hope all of you will now unfuck your custom clients.

>>108670334
Qwen 4 will achieve 77 on the bench

>>108670252
Orb is written with python and javascript.

>>108670354
It is the definition of webshit application then.

>>108670252
It has a backend just like ST. How do you think these frontends store your data?

>>108670334
Why do you think we havent already? How do you explain the 7trillion dollars invested into us ai companies, 1 year ago? How do you explain a massive military clamp down on yhe global oil supply, restricting china's access to oil?

>>108670378
I don't know. If it is so easy why it isn't there already? Automatic chat log export.

>>108667965
thanks anon, I've been used to sillytavern after using it for years, I'll follow the sillybunny fork

>>108670361
You kind of sound like an idiot. It would take like 3 lines of code to produce a file for the user in python or javascript. Being a "webshit application" has no bearing on that functionality at all.

>>108670387
NTA, I hate webshit, but you should really learn how webshit works before criticizing it. Makes you look silly.

>>108670279
Way better than gemma4 31b, I don't know how they did it but this fucking thing is almost the same as the current sota models at coding.

>>108670485
+5RMB

>>108670443
Maybe so... I wanted to present this idea because I enjoy a debate.
Is it really so?

Why are Google's model so fucking dog shit when it comes to coding

>>108670354
Are you orb anon? and if so that's extremely embarrassing that you think exporting chats is not possible or hard.
The absolute state of vibeshitters.

>>108670517
No, I'm not him. Also I'm saying it should be easy.

>>108670513
they don't want to distill opus

>>108670513
>>108670535
Is it possible to avoid this by using RAG then?
I think most of the proprietary models are utilizing database knowledge too but it's not visible to the end user.

>>108670354
>python and javascript
Cool. We can get hacked from two different sources...

>>108670554
python and javascript are the future whether you like it or not

>>108670554
You should definitely turn off your computer to not get hacked

>>108670554
where is your pure assembly front end then bro

Happy Thu(rin)sday

>>108670560
There's no reason to use Python for a web backend. It barely has a reason to be more than an HTML file.

>>108670580
keeping it to myself where it can't be trained on

>>108670485
>almost the same as the current sota models at coding.
OK nice try, you lost me there.

>>108670605
My frontend is pure javascript and runs entirely in the browser. everything is stored with pglite in indexdb. It's essentially a static page.

File: file.png (594.6 KB)

594.6 KB PNG

>>108670603

yjk

>>108670708
I hate what this site did to me...

>>108670165
My lust has been provoked.

>>108670708
hmmmm

>>108670716
need to train a xxzero style and get on this

File: Screenshot 2026-04-23 at 21-09-23 Orb.png (189.8 KB)

189.8 KB PNG

Why does the inspector say more fragments are activated than I have picked?

>>108670784
because it's vibe coded.

>>108670784
Seems like a loop leak.

It's crazy anons have to use vibecoded frontends because the current ones are so shit

>>108670822
Models have gotten better than the average bootcamper and hobby devs.

>>108670822
ST is perfectly fine. specially with gemma.

>>108670381
>7trillion dollars
was an ambitious sama goal. in the end openai has "only" raised 200bil so far.
>military clamp down on yhe global oil supply
oil is mostly irrelevant for ai

>Why do you think we havent already
because the people at ai companies are still working. agi will make them obsolete first

>>108670822
have you seen Kobold's dogshit frontend

>override-tensor = "blk\.0\.ffn_.*=CPU"

[55363] error while handling argument "--override-tensor": unknown buffer type
[55363] 
[55363] usage:
[55363] -ot,   --override-tensor <tensor name pattern>=<buffer type>,...
[55363]                                         override tensor buffer type
[55363]                                         (env: LLAMA_ARG_OVERRIDE_TENSOR)
[55363] 
[55363] 
[55363] to show complete usage, run with -h
[55363] Available buffer types:
[55363]   CPU
[55363]   Vulkan0

wtf

>>108670822
ST is fine. Although I feel vibecode ripping out some unneeded copium parts that only made sense on pre-gemmy models.

>>108670888
>>108670856
Still lacks good ux and features
>>108670851
I would say for local gemma is the "X" factor, I expect more bespoke projects for things to pop up. I think what kills most of the mainstream frontends are how overly opinionated they are which makes people annoyed. Also these vibecoded frontends are incorporating all the features while taking the easy wins.

>>108670886
Remove the .*, see if that does anything.

>>108670900
>features
such as?

>>108670910
Nope, not even "blk\.0\.ffn_=CPU" works

It's a shame but I went back to Kokoro. It's fast and light even on CPU, it supports many languages, and its pronunciation is... fine. What I did to solve the mixed language use case is to simply just detect language segments and route them to the voice that works in that language. And have the audio queued up. This does mean that the voices change for each language in the input, but for my use case I don't require an immersive experience.

I integrated this into my voice control app, where I can now highlight a piece of text wherever and say "read" or "pronounce" and it will read it out for me. We are so back.

>>108670913
In my case a good all in one RAG solution that's not a outdated extension that performs like dogshit.
I don't know about the RP anons but I think there's a ton on the table to improve things and I might take a stab at a proof of concept

>>108670886
you have double "=" characters

>>108670038
I sure hope it won't be yet another example of an anon revealing something cool and then disappearing

>>108670942
https://github.com/ggml-org/llama.cpp/discussions/13154

File: llada2.0.png (1.1 MB)

1.1 MB PNG

This should also be of interest here.
https://huggingface.co/inclusionAI/LLaDA2.0-Uni
Multimodal image generation+edit but also text diffusion (yes text diffusion) model.

>>108670971
I'm setting it in the ini template per-model

>>108670998
>lada-mini
This has to be a joke what most UK/US posters don't understand.

>Qwen3.6 dense out

is it time to buy more VRAM?

DeepSeek's web chat just changed its system prompt because of that anon from the previous thread lmao. It seems like it has more instructions now, judging by the thinking.

Now It's been confirmed that DS labniggers browse /lmg/

>>108671048
@grok what is xe talking about?

>>108671048
>because of that anon from the previous thread lmao
that was?

why do people use fish audio? the tags barely change the speech output at all. [whispering in soft voice] for once sentence and [shouting] for another sentence still makes them sound basically the same rather than being truly expressive.

>>108671070
>>108671062
>>108663630
Say something you want them to know

>>108671048
1- if something is written here, it's probably written in reddit, twitter and discord
2- why the hell do you use the web chat

>had sex with female character in medieval setting with gemma 31b
>she asks if I used protection
I like gemma but I really want that 124b now.

>>108671096
She didn't ask that.

>>108671096
Sir your sheep intestine?

>>108671096
if you didn't use the thinking process that's on you

>>108671096
even sota models do that shit, at least the ones I tried in 2025
only way to bypass is to have a second model do an anachronism check

>>108671096
the 124b would be 10b active so a third as smart