we need a ponified 4chan extension/userscript

>>42970396
I agree.

File: 1930833.gif (224.3 KB)

224.3 KB GIF

>>42970396
I can only assume that kind of thing is horribly complicated coding

>>42970802
>I can only assume that kind of thing is horribly complicated coding
4chan-xt is (roughly) 26k lines of TS code

Slow start.

Quad9 anon here. In case it hasn't been addressed yet, I had a chance to swap to the unfiltered DNS ahead of a network reboot and can confirm Foalfetch is loading through the unfiltered DNS. It's just the filtered options being stupid again.

File: 1679232546959.jpg (112.4 KB)

112.4 KB JPG

>>42970929
I THINK 4CHAN-XT SUCKS

>>42970929
>>42971994
It's also abandonware. Retvrn to X classic or perish.

>>42972283
>Retvrn to X classic or perish.
automatic webp2png is a must-have feature
>It's also abandonware
ponefork when?

File: bee da change you wanna see.jpg (337.5 KB)

337.5 KB JPG

>>42972299
>webp2png
WebP is one of El Goog's worst sins, but I get by fine with Right-Click -> Copy Image -> Ctrl+V.
>ponefork when

>>42972331
>but I get by fine with Right-Click -> Copy Image -> Ctrl+V.
fuck I forgot about clipboard saving images as png kek
alright then I need big image conversion to jpeg

>>42972331
>bee da change you wanna see
I'm not a JS nigger and both -X and -XT use some gay frameworks (afaik they make shit even worse)

>>42972331
No idea why everyone is so afraid of WebP, other than that it's just different from what they're used to. The only thing I hear is "NOTHING SUPPORTS IT!!1111" but that seems more a fault of the software those people choose to use and/or their own lack of desire to update it.

>>42972427
Maybe because it is yet another internet component that Google controls?

>>42972427
WebP's improvements are too minor to consider upgrading everything
VP8 was based because lol free codec
JXL is based because literally JPEG 2.0, you don't need to change to support it

WebP is a fucking video codec turned into a still image

>>42972447
>change
*change much

>>42972427
Because its forced usage gives me image files that PRETEND TO BE OTHER FORMATS UNTIL SAVED

I wish Ponepaste had Italics and Bold.

>>42972427
Because there is no reason to use it, and Google really wants everyone to use it, therefore I do not want to use it.
And every website is pushing it really, really fucking hard. Previously, websites that would recompress your image were considered garbage, and if you uploaded a png somewhere you could expect people seeing it to be able to download a png. Now, 90% of big tech websites have decided that if you upload a png, people will see a link to "image.png", they can open it in a new tab to see a url ending in "image.png", and when they download it they will get a webp. Just fucking changing your formats has suddenly become normalized out of nowhere just because muh webp, muh google format.

Software is only at fault when it actually supports webp, helping propagate this google virus of a format. Software that does not support it is functioning correctly.

>>42972460
That's the fault of sites implementing it wrong though.
>>42972522
The reason is to save storage space, make sites load faster (especially on slow connections,) and save data transfer quota.
Being a Web ad company, Google really wants everyone to use the Web. Therefore, by your own logic, you should not want to use the Web, and you should unplug your computer and go to bed.
Also, WebP can be lossless - see once again, sites implementing it wrong.

>it's another WebP advertiser episode

>>42972522
tl;dr >i have autism and i can't handle change therefore any change is objectively bad

>>42972542
webp's usecase is compression at all costs, including quality and software support

>>42972532
>WebP can be lossless
PNGs are betrer for this

>>42972582
https://www.ctrl.blog/entry/webp-flif-comparison.html
>Compared to the reference PNG, WebP’s median file size reduction was 24,3% and the 85th percentile was 16,6%. Only 0,03% of the WebP images were larger than the PNG.

>>42972595
>limited image resolution

>>42972508
Did it get any updates at all?

Webp is old school, AVIF is the new hotness

>>42972532
>That's the fault of sites implementing it wrong though.
Pretty much nobody would actually upload webps anywhere if sites didn't forcefully auto-convert it, so I don't know if I buy this as an argument.
But sure, in theory if it was a completely optional format anyone could choose to use or not to use, I really wouldn't have nearly as much issues with it.

>The reason is to save storage space, make sites load faster (especially on slow connections,) and save data transfer quota.
It's not the 1990's anymore. Loading a 500 kB PNG rather than a 450 kB webp won't kill you or your connection, nor your storage servers.

>Google really wants everyone to use the Web
The web existed before google and will exist after google. Google also mostly cares about you using google products specifically - they want you to watch youtube, search using google.com, sign in to google.com, use google docs, and most importantly - whatever you do, do it in google chrome. The rest is mostly irrelevant to google.

>webp can be lossless
Speaking of actual technical issues rather than ideological ones, this is a big one actually now that you mention it. Webp can be lossless. Webp also CAN be lossy. When a website serves you a webp image, there's zero indication that you're actually downloading the lossless original, absolutely ZERO; so the fact that it maybe could be lossless, if the website feels like it on that day, is really not useful at all to anyone.

>>42972915
*JPEG XL. By now it's supported by default by MacOS, Linux Mint/Ubuntu/Fedora, pretty much all FOSS/freeware image editors. Safari support is already there, and Chrome and Firefox are coming.
(Unfortunately, Windows support requires official extension from MS Store, so not on by default)

>>42970396
We already have userscript for pone waifu flags:
https://dl.dropboxusercontent.com/s/lxgwst61d6o9xik/Improved%20Waifu%20Flag%20Selection.user.js?dl=0

>>42973127
>using flags
>ever

>>42973112
>there's zero indication that you're actually downloading the lossless original, absolutely ZERO
You can tell by looking at the format with one of the official WebP tools (webpinfo or something like that). Sure it's not exactly user friendly, but it's not *impossible*. Still doesn't excuse the use of the format for original copies though (I'm fine with people using it to serve thumbnails or other quality reduced versions of the image, as long as I can get the original quality image in a regular format, preferably one not controlled by Big Tech).

>>42973165
trying too hard here, mate

>>42973112
>It's not the 1990's anymore.
fun fact: a much larger number of people have a slow Internet connection today than in the past, simply because so many more people have access to the Internet AT ALL. There are millions of people who's only choice is dial-up.

>>42973346
bullshit, poorest countries in the world shitpost on 3G with their phones

>>42973346
>a much larger number of people
subcontinentals don't matter, outside of being an invasive species

>>42973424
>>42973441
hey, if yall wanna be racist and make your site hard to use for people from certain areas of the world (including parts of the US,) that's entirely up to you. personally I've chosen a different path in life!

>>42973489
Zigger. Zigger zigger beaner. Dilapidated Farm Equipment.
Get back in the orchard.

>>42973493
love you too anon <3 i hope you have a great day!

>>42973489
>3G is racist

File: +1776 帕兰提尔评分.jpg (51.2 KB)

51.2 KB JPG

>>42973520
You underestimate just how terrible American internet can be.

>>42973607
get starlink or smth

>>42973183
Yeah you can always inspect it after the fact, of course, but my point is that as the website is showing you the picture and as you're downloading it in your web browser there's no indication.
If they had named it like .webpl for lossy or something, then it would at least have mitigated this issue.

>>42973489
>personally I've chosen a different path in life!
You've chosen to operate a website that auto-converts user images to webp? Good for you, personally I've chosen to never use your site as a result.

>>42973717
I think he means how Philomena webpages are 200 Kilobytes each. That's like half of a novel for an almost empty page. The size doubles if there's higher than average amount of tags.
Good for userscripts, a waste for everyone else.

>>42973862
Though I guess gzip/zstd lowers that size significantly

>>42973489
>hey, if yall wanna be racist and make your site hard to use for people from certain areas of the world
Please webmasters, do exactly this.

>>42973717
>If they had named it like .webpl for lossy or something, then it would at least have mitigated this issue.
Wouldn't really have helped, since file extensions don't really exist on the internet since everything works with MIME types. That's why I get served PNG images from one site and it isn't until I download them and open them in my local image viewer that I see it's actually a WebP image.

>>42973862
What does that have to do with the image formats

>>42973987
The fact that websites mask the URLs is also an issue, but even an honest website showing .webp links can't signal the lossless/lossy distinction - that's a fundamental limitation baked into the format since it inherently always supports both.
For instance, PNG and JPEG are generally never mixed up, because website simply serve PNG-type images under "image.png" URLs and similarly for JPEG and everything else.

It would also help once you actually download the image and it saves a webp file on your computer - you'd at least see if your browser is saving a webp or webpl file.

>>42974022
I have a large images folder with pretty much everything I've ever saved from the Web, started long before WebP became popular... I wrote a script probably a decade ago to change file extensions to the actual type based on the MIME type, because I have so many PNGs with .jpg extension, or the other way around. I also wrote a tool to detect JPEG compression in PNG images, and discovered I have a ton of JPEGs that were re-saved as PNG.
This confusion is not a WebP problem. It's a Web problem.

>>42974111
>I wrote a script probably a decade ago to change file extensions to the actual type based on the MIME type, because I have so many PNGs with .jpg extension, or the other way around.
Irfanview warns about this and lets you change the extension when you open the image which is nice, I haven't run into this all that often though.
>I also wrote a tool to detect JPEG compression in PNG images,
That's pretty handy, especially with the right-click -> copy image turning it into a png issue that is plaguing the internet these days.
What did you use? I saw someone mention something called brisque but I haven't really looked into it for detecting jpeg artifacts.

>>42974111
>I also wrote a tool
Trips of user friendliness.

File: image_2024-08-21_01-34-13.png (66.9 KB)

66.9 KB PNG

>>42974183
wrote a rust script for it; it looks something like this (i say something like because there is another copy on my computer that probably differs from this; i'm just not on that computer) https://gist.github.com/AppleDash/ec3ad93819febdc65e9c0864d2afa595
i am too lazy to actually go through and look at what it does, but if i recall correctly it decodes the images and then calculates the coefficient of variance of the chroma channels across 2x2-pixel blocks of the image, with the idea being to catch the macroblocks.
line 70 is what would do the actual detection; i think the version of the code i linked is modified to run on a batch of images and generate the picrel graph, which was based on 10k random images from Derpibooru that were known to be jpeg or png format. below/above a certain threshold for the CoV, it becomes more likely that an image is a JPEG vs a PNG; you can see it works pretty well.

File: 1738831159451670.gif (429.8 KB)

429.8 KB GIF

>>42974308
>rust

>>42974323
the irony of being a tamersfag and yet complaining about things

>>42974327
you got the joke

>>42974323
good news is it's 2026 and you can just paste it into chatgpt and say "rewrite this in my favorite language"

>>42974308
That works for most JPEGs, but not all. JPEGs can also be compressed with 4:4:4 subsampling (not sure if it also supports 4:1:1, I know some video codecs do, along with 4:2:0). Such a tool won't detect these cases. They're probably quite rare though.

>>42974348
if you can find an example of one, i'd love to see if i can make something that works with it!

>>42974353
I don't have any examples myself I'm afraid, I just remember I had a tool (ages ago, it probably doesn't even exist anymore, this was like Windows 2000 time, IIRC) where I could select the level of subsampling used and also let the program analyze the image to automatically mark certain MCUs to be compressed more strongly (where there was little visual information, such as a sky in a photograph) than other areas of the image. Pretty interesting tool, but unfortunately in my testing I found that Photoshop's standard JPEG encoder was actually better quality at the same filesize (well, at least until their "updated" encoder in version 6 which was significantly worse).

>>42974353
According to paint.net this one should be compressed using 4:4:4 subsampling.
https://u.pone.rs/acrmdaav.jpg

>>42974384
Oh I think I remember: it was called JPEG Wizard if my memory is correct.

File: twalaight.jpg (212.9 KB)

212.9 KB JPG

>>42972758
hell no
well actually to tell the truth I have no idea if it got any updates
but fuck no it didn't get any fucking updates bitch
the fuck's wrong with you

>>42974415
It didn't take you that much time, did it.

>>42972508
See >>42947412

>>42974353
(Oops, I forgot that 4chan re-encodes images. Here's the original:)
https://u.pone.rs/wynjytqb.jpeg

>>42975147
>>42974394
interestingly, my code detects both of those as JPEGs!

>>42975639
Does it detect original image as PNG?
https://cdn.twibooru.org/img/view/2020/7/9/5.png

>>42975642
it does! this is very interesting to me.

Is "Fillymaxxers Anonymares" floorb's youtube channel

>>42976256
no thats huffy
dude is having another meltdown over shit he imagined

>>42976296
Cringe, he's giving fillymaxxers a bad name

>>42976296
dude is based

>>42976298
all fillymaxxers are based
>>42970348
so, it's been like 3 years since i deprecated the Twibooru v0 API by making it fail 50% of requests - is anyone here still using it? you can identify this by the absence of "/v3/" in the api endpoints you are using, instead the endpoints look like "/1234.json" or "/search.json" or "/tags.json" etc.

>>42976355
>i deprecated the Twibooru v0 API by making it fail 50% of requests
Why?

>>42976664
why deprecated?
because it's a lot of extra code to test and keep track of and it sits directly in the main code path and basically every endpoint is like
if (api) {
do one thing
} else {
do another thing
}
and that's annoying and hard to maintain
why that method?
because it would alert people that don't read this thread as to what was going on (the error message says to use the new API)

>>42976718
>yanderedev worked on Derpi
That explains a lot.

>>42976718
>(the error message says to use the new API)
Oh okay, that makes more sense. I thought it was just randomly failing every other request in some sort of spiteful way to punish people for not updating to the new API.

>>42976355
>fillymaxxers
You really go in with this meme, don't you?

>>42978878
what meme, anon?

>>42978878
"fillymaxxers" are just /mlp/'s girlmoders

>>42979202
>girlmoders
Who?

Hey anons, this is a bit more out there than the sites usually discussed on this thread, but me and another anon have been working on a con tracker/archive since all the existing ones are either dead or leave a lot to be desired
We only put it online a couple days ago so its still very bare-bones, incomplete, and leaves a lot to be desired, but any suggestions or feedback would definitely be appreciated
mlpcon.info

>>42979588
>Not calling it con.me

File: only the good die young.png (541.3 KB)

541.3 KB PNG

>>42979588
Very nice. I do want to point out that your Google Sheet has entries for FillyCon 2018/2020 and High Roller Pony Con 2019, none of which actually happened. Also, what's Endless Forest Camp South? I've never heard of that one, and I can't find anything about it online.

>>42978904
The filly meme.

>>42980183
>I do want to point out that your Google Sheet has entries for FillyCon 2018/2020 and High Roller Pony Con 2019, none of which actually happened
Most of the past entries are based on info from an old archival spreadsheet called the My little pony convention & event spreadsheet of doom. It a couple old cons on it that were announced but didn't happen which is where those come from. We left them on the sheet, but off the website while we decided between not display them at all or listing them separately for fandom history archival purposes, even if they didnt actually happen
>Also, what's Endless Forest Camp South?
Chinese MLP camping event. Its pretty hard to find since even if you use quotes, it has to match up with the google translated text.
https://www.equestriacn.com/2023/02/2023-endless-forest-camp-camp-campaign-publicity.html

>>42981909
there's no "botnet", that's not how the Internet works. a random site can't magically "use your IP" just by you going to their site.

>>42981909
Why not? I'm not doing anything illegal. And even if they are I have plausible deniability: everyone else is using it just to bypass bullshit 4chan bans as well.

Anyway, just came across this on Desuarchive:

>This post was modified by Desuarchive Moderator on 2018-08-02
Never seen that before, the post didn't even have an image attached that was removed. I wonder what it was that was so bad it needed to be deleted, the post was literally just a single-line reply to a "what if" scenario.

>>42981878
Well that's a new one.

is twibooru under attack or something?
page load times are 10+ sec

>>42983030
Works on my machine. Maybe you're shadowbanned or something.

>>42983030
can confirm we were under attack by what for all intents and purposes looks like a botnet intentionally hitting intensive endpoints - but in 2026, is more likely to be a rogue machine learning training scraper. i've deployed a mitigation to try to improve the situation.

File: pony rocks.jpg (856.2 KB)

856.2 KB JPG

>>42981976
>a random site can't magically "use your IP" just by you going to their site.
If I'm not mistaken, website A could send requests to website B if CORS headers on website B don't tell user's browser not to do so. I wouldn't be surprised if some website A hosted by "edgy" people abused some misconfigured website Bs that way, leveraging its own users.

>>42981994
desu removes cp and doxxing

>>42983631
And leaks, don't forget leaks. It doesn't matter if you're a multimillion corporation or a circlejerk of b&thro artists, they're slobbering on that DMCA boot so hard they're shitting shoelaces.

>>42983860
Nah

>>42983860
I've only ever seen gore or dox getting removed (never came across CP or it had already been removed when I got to the page). That's why that edit is so weird. It was just a guy going "I'd get up and leave" to a what-if scenario in the OP. No attached image that was removed so no CP or gore and I doubt the post contained dox in its original form. So why the edit? The only thing I can think of that makes sense is the email field was set to blank by request of the original poster (it was some namefag).

>>42984124
They also removed the ASOS download links when their dumbfuckery ruined a large pony archive.

Any new horse stuff?

So does anyone back up /mlp/ pictures from desuarchive&co?

>>42988362
I don't back up anything anymore. Now that I have AI I can create whatever I want anyway.

>in b4 trolling
Besides the old stuff that has long since been backed up, the nu-fandom's output is garbage I want to stay far away from.

>>42988778
>the nu-fandom's output is garbage
That's not entirely unfounded, you know. When you see the new entries on the boorus, you'll notice a pattern when it comes to quality.

>>42989141
Pick up. The pencil.

>>42989437
That won't do any good. Trust me, I tried.

>>42989730
The guitar pick then. Or the shape-sculpting tool. There's gotta be something you're not crap at.

>>42989871
Yeah, it's called proompting. DWI, anon. You don't even have to worry about seeing my gens 'cause I don't upload any of them.

>>42989883
If a tree falls in a forest and no one is around to hear it, does it make a sound?

>>42990008
I don't care about other people or the fandom, as long as I get what I want. AI gives me what I want. Simple as.

Unrelated but I think 4chan is just about killing itself except as a place for terminally online autists (like us) with their new Craptchas. I don't think any normalfag who is used to just pressing the Recaptcha button wants to solve not one but THREE puzzles just to make a post, not to mention first waiting a minute before being able to request one in the first place.

Which is somewhat good for us because it means 4chan will become uninteresting for normies and go back to its roots again: to be a haven for misfits and people pissing the time away. Sadly, nowadays among these we have those that don't even fit in among the misfits themselves, so we're stuck with negligible quality of posts after all.

File: 1762290261963478.png (135.2 KB)

135.2 KB PNG

>>42988778
>>42989883
So which AI shartup are you invested in and/or an employee of? There's no other explanation for why you'd go out of your way to shit on all human creation and advertise generative slop machines unprompted in a thread about alternate sites and backups.

>>42990365
>offended despite me saying I'm not even publishing my gens
Is that your coommission money I hear drying up?

>>42990654
Usecase for posting about AI in the archival thread?

>>42990671
I gave my reason for no longer backing up new content (which is mostly garbage anyway). The fact this offends you means little to me.

>>42990138
>their new Craptchas
They may ward off the normies, but they sure as fuck don't stop the bots. At least not for long.

>>42990718
Usecase for clogging archival threads with refusal to archive?

>>42990746
The thread is getting clogged with your incessant nonsense far worse.

Polite herbs.

>>42990767
Usecase for assuming everyone disagreeing with you is one user?

File: thousand mare stare.gif (146 KB)

146 KB GIF

>>42988362
I'd like to, but
>this is a place where odd people come to post
>odd people can be the cool kind or the uncool kind
>the uncool kind could post bad pictures
>jannies try to delete things but some might get imported to desu early
>I don't want to automatically download a bad picture accidentally
>I don't have time to manually download pictures, but that would help ensure I don't download bad pictures
I thought about downloading only 4chan's ephemeral archive, assuming everything that lands there has already been moderated, but I'm not 100% sure of that. I suppose this is a somewhat considerable concern for archivists of any website with user generated content, how's one supposed to handle this?
I just want to help pony history survive in redundant places, not get tangled into this mess of bad actors.

>>42990883
>how's one supposed to handle this?
You're not. You're supposed to give up and let Big Bro handle what you want to do. That was the entire purpose of making the "bad material" you speak of the big scare in the first place.

So either you say "fuck you I don't give a shit about fucking puxels" or you do as they say and be a good boi and lie down.

AYO TO THE NIGGA RUNNIN PONERPICS:
fix yo shit cuh that shiet be goin down to internal server error. also yall better be checkin the site n shiet cuz it go down all the time goddam how yall aint noticin fukin lazy ass niggas

>>42991510
keep in mind all the fan sites are run by people in their spare time for free, and the nice thing is we have so many alternatives so if one's down you can use another!

>>42991764
nigga the "alternatives" are missing images from the other "alternatives." i dont even have my favs. a nigga cant even jerk off to floots without gettin a internal server error

>>42991510
The admins haven't been heard from in months. I think they're down to a single volunteer watching over the place.

>>42991510
Hey buddy, I think you've got the wrong door. The SHITTED booru's two blocks down https://twibooru.org
>>42991810
>a single volunteer
Literally who?

>>42991904
>Literally who
I'm assuming he's speaking of Mold Geddle.

>>42991904
Gold Meddle is still alive as far as I know.

>>42991510
We notice, there's just very little we can do with our aging codebase.
>>42991938
Don't remind me.

>>42992068
>aging codebase
Dude, you literally ported the site to new Philomena version, then reverted to old version because of Internal Server Errors.

>>42992111
The software stack being old makes updating individual components troublesome. You can't just magic in a new search implementation without running into incompatibilities.

So is the fact none of these boorus ever get updates really because it's so much work, or because the admins have no spare time between their three jobs to spend just an hour coding?

Because if they're like me (hikki NEET losers) then I don't see why they can't spend just a few hours of their day to fix shit, when some of the problems can literally be fixed with updating just a couple lines of code (in b4 "you do it then" I would but I'm a codelet/general brainlet, I can just barely understand that the problem requires very little code to be fixed, but I can't oversee the entire picture, but any of you code wizards definitely can).

>>42992256
>it's easy
>it's hard
So which is it?

>>42992438
Both, dummy.

File: Screenshot_20240216_204311.png (697.4 KB)

697.4 KB PNG

>>42992256
>tfw all tech jobs are just a plot by Big Tech to suck away energy that could be spent on pony tech

>>42990883
Your problem is literally pulled out of your ass. The most critical stuff in the danger of deletion by desu is older stuff. Just don't archive pics newer than a year-old if you're paranoid.

>>42992956
To elaborate if you didn't understand: desu highly likely already deleted all illegal pictures by the time the post was up for one year.

>>42992246
It's interesting to note that Derpibooru 5 years ago, running the same codebase you are today, being 10x more popular, did NOT have constant Internal Server Errors. It seems to be a problem with the site / configuration rather than the code.

kemono seems to be going under
how should we archive the pony creators from there?

>>42993527
FUD unless you have SEEEKRIT HIDDEN INFO you're willing to elucidate upon, plenty of import updates from today (whether there's new content or not).
Their only issues are Substar being exceedingly hostile to automated access of any kind and people trying to triple-layer paywalls.

>>42992438
Easy for you, hard for me, obviously. More accurately: it's easy in the amount of work involved (time, which I have plenty of), but hard in the brainwork required (in which I'm sorely lacking). But not for you people who coded the whole damn thing. If you wrote 1,000s of lines of code just to get it to release, why are 10 lines of code such a problem now?

>>42993545
Check the sister site (coomer). Nothing new is imported since the end of october. Plus is full of spam. Millions of spam posts across any and all account just advertising offsite stuff. These are not present at the source. Not AI slop spam on coomer.
Kemono is getting AI gen spammed too via thousands of free accounts. If you check the recent posts, 99% are aislop spam.
Regardless, it would be good to have a pony only alternative.

>>42993595
>kemono is falling
>why
>OTHER SITE
Yep, FUD. Shut up.

>>42993595
>Paytrolls use aislop to try to protect their coommission money
Pottery

>>42996071
You sure it's not from an alt?

>>42996597
derpibooru_id search on twi gives that image when using that ID number so i'd say he's on the money.

What would a website ran by ponies be like?

>>42997794
There would be a lot of horsing around.

>>42997794
The server would run on horse power.

>>42997818
Not to be confused with whorsing around, which is what sysadmins have to do when the bills threaten to overwhelm them.

>>42998089
Elaborate.

File: mfw.jpg (82.1 KB)

82.1 KB JPG

https://infrablog.lain.la/pomf-announcement
FYI, since plenty of ponies were posted there. I hope pone.rs anon has measures in place to combat this shit.
I don't know what to say. It's all so tiresome.

>>42998839
>yet another leftard burning their shit down over Infinity Illegals
unfortunate but expected, hopefully someone can rip the pony content from it.

File: L is for Libshit.jpg (143.1 KB)

143.1 KB JPG

>>42998839
KWAB

Looks like progress on boorus have stalled, and entered maintenance mode (two alt-boorus even died). What needs to be done to rekindle the enthusiasm?

>>43000796
It seems like everyone's become extra busy lately. Me included, although perhaps more extra tired than anything else.

>>42998865
Based retard. But that's what you voted for after all - to let an even bigger retard be your boss. Democracy really is a country's undoing.

>>42998839
>It's all so tiresome.
It really is. Especially because as the account of this owner once more shows these sites are being actively targeted by "someone" and Big Tech companies get held the hand over their head for "some" reason.

>>42998839
Good riddance. Lainfags are a plague, and I'm not saying that just because this one blocked my entire country.

>>43001894
What third world country are you from?

File: muff hunter.png (2.1 MB)

2.1 MB PNG

>>43001999
Checking those trips, I'll give you a big hint: there's a good chance we'll be embroiled in another war in the coming weeks.

>>43002025
That doesn't really narrow it down much you know. Anyway, hope you're doing okay.

>>43002025
Greenland?

>>43002070
Thanks, nonny. I'll hunker down somewhere safe.
>>43003628
You got one hemisphere right.

>>42992256
I can only imagine it's because they were in college when they wrote the software and now that they have fulltime jobs they have too little time left to keep updating it. Which means they should hand over development to another capable person who does have enough free time.

>>43005633
I mean, you're not wrong about the free time part. But it's not like there's a whole bunch of people clamoring to get involved either.

>>43005708
Kind of hard to get involved when the codebase for Twibooru, for example, is hidden away in some link in this thread, with not a single reference to it on the site itself, not even on the "source code" page. And telling people to mail you or send you a message to get involved isn't exactly welcoming.

>>43005824
What also pisses me off (I mean nobody here is going to listen but I'm still going to say it) is that there are many pages that have broken links or outdated information that could EASILY be updated by someone that doesn't know how to code. The contact page for example links to an IRC chatroom that no longer even exists. Something like that can easily be updated. Are the people in charge of these sites even still interested in running the things? If you're too "busy" to do much work, at least let other volunteers fix shit besides the tagging on some pictures.

>>42998839
So is anybody here planning to scrape Desuarchive for Pomf links to back up? I'm willing to help if it involves running a script or something but I'm not capable enough to write the scripts myself.

>>43006025
are you volunteering as Twibooru staff to do those things? feel free to contact me or Twifag on Twibooru if so.
>>43006299
i suppose i could investigate that.

File: 2730268__safe_artist-colon-egor418_derpibooru+import_twilight+sparkle_pony_ambiguous+race_bust_female_horn_image_lineless_mare_minimalist_modern+art_pn.png (269 KB)

269 KB PNG

>>43005824
If people are looking to get involved, I've got something anons could help out with. I've been wanting to make a new captcha for NHNB and I could use some help drawing some of the new elements for it. Specifically I'd be looking for two-toned simplified SVG profile shots of the mane 6 + Celestia and Luna. Black and grey, with one tone for the mane and one for the body. And by simplified, I mean easily identifiable at ~50-80px tall. Roughly square dimensions. Do be warned that even at the best of times I work at a glacial pace, so it'd be best to expect any results to be delayed a while. I was planning on doing this myself, but I figure I can throw it out there to see if there's any interest (and also due to the glacial pace that I've been moving). I've got a few other things, but I'll start with this and see if there's any takers. If not, no big deal.

>>43006939
>If you're too "busy" to do much work, at least let other volunteers fix shit besides the tagging on some pictures.
guess.

>>43005824
>is hidden away in some link in this thread
That link is pointing towards woefully outdated repo from literal years ago, anyway

>>43007148
How do we contact Twifag or you if the need arises, anyway? Matrix went kaput, AFAIK, and other Anon mentioned that IRC is gone, too.

>>43008023
Email admin at the domain name works (we get tons of emails there)
Also, can send DM on twibooru

File: 1756042650778778.png (162.9 KB)

162.9 KB PNG

>>42998865
So this is the intellect of /mlpol/. No, he's taking it down until he can get a solution to the problem of cp spammers using his service to host their garbage, which is the same shit that killed smutty. Of course, you wouldn't know that, because you're probably only ever here to preach what retarded ideals your psyop board fed you. Also you clearly didn't actually read the blog detailing everything, because more than 140 characters makes your brain hurt.

>>43001894
Maybe your country shouldn't be a fucking problem then, eh?

>>43008416
>/mlpol/
haha, you wish.
>NO IT MUST BE PSYOP RETARDATION REEE
>'The weaponization of the US DOJ and subsequent character assassination against anyone who goes against their regime or is otherwise convenient to scapegoat.', followed by multiple hit pieces (some paywalled) and literally nothing else.
Reason 2 listed, above the pizza with no toppings you hyperfocused on. Crossreference your own shit next time.

>>43008023
irc isn't gone, the web client is down. however, the most likely person you are to get in contact with there is me.
>>43007459
it's not, and what >>43005824 is saying is kind of irrelevant, for a few reasons - it's not on GitHub for complicated reasons. and even if it was, it would be very unlikely that we would blindly accept "pull requests" without talking to the person making them first anyways, because it's extremely unlikely that they would find themselves in a position to test the code first, which would move the burden for that to me, which is just not something i have the time for. i have the time to sit down with someone for half an hour and tell them how to get it running, in the hopes that over the next year they might do more than i can do in half an hour myself, but so far of the 3 people i've done that for, all of them failed to produce a single line of code.
i was trying to make this a bit easier, but once again, no fucking time or energy.

>>43008525
Explain it to me, I'll record your voice, generate transcript and put it in README with screenshots.
(Not right now, tho. I'm busy with another project. Making it in C++ was a mistake)

>>43008559
i mean i could make a screencast of me setting it up lmao
but not right now because i have laryngitis and i am not speaking until it's better

>>43008569
Suck less cock.

>>43008629
friend - if the cock, or any emission therefrom, is contacting your larynx, you are doing something very, very wrong.

File: 986854.png (1.9 MB)

1.9 MB PNG

>>43008525
>it would be very unlikely that we would blindly accept "pull requests" without talking to the person making them first anyways
Alright, that would explain why LTO tapes are the only medium that can fit twiggy booru source code.
>tell them how to get it running
Does Twi have some non-standard deployment stuff that a moderately competent anon couldn't figure out on their own, equipped with access to the internet?

>>43008785
>Does Twi have some non-standard deployment stuff that a moderately competent anon couldn't figure out on their own, equipped with access to the internet?
extremely
i think most people would just give up without direction, but i might be underestimating people
i know that if i didn't already understand it intimately today, i would probably give up trying to set it up.
off the top of my mind, you need the right Ruby version, a bunch of random system libraries for Ruby gems (eg: libpq-dev for postgresql, just a random one off the top of my head); a special fork of FFmpeg installed in /usr/local; a first-party library obtainable from Twibooru's GitHub organization (libmediatools) that you have to also install into /usr/local; the right version of imagemagick; a bunch of random tools like rsvg-convert, gifsicle, jpegtran, optipng... and then you need postgresql and elasticsearch both with a non-standard/non-default configuration, and redis in its default configuration... that's what i can remember right now.
this was relatively up to date at one point, but now it's almost 4 years old so it probably won't even remotely work: https://git.twibooru.org/Twibooru/.git/tree/doc/deploy/README.md but it gives you an idea of what's involved.

>>43008420
How is he wrong?

All lain.la links from cytube replays:
https://u.pone.rs/srzabpfd.tgz

>>43008991
based, thank you - you planning to get them from desuarchive as well? i'll download the links...

>>43008806
There's a reason I decided to write my own booru engine rather than use Philomena. Which I still need to finish.Which I know is different from BoR.

>>43009458
philomena is actually much easier to run in development because it was made with that in mind. it's the production deployment that's near-impossible!

>>43008806
>https://git.twibooru.org/Twibooru/.git/
>.git
Is your repo not bare? Typically server repos should be bare (twibooru.git). Here's how to convert non-bare repo into bare:
https://stackoverflow.com/questions/2199897/how-to-convert-a-normal-git-repository-to-a-bare-one
https://stackoverflow.com/questions/3382679/how-do-i-update-my-bare-repo

>>43009916
It /is/ bare. So sayeth git.

>>43009916
i guess it's not, but the process for making that repo in the first place is annoying and i don't necessarily want to break it. does it matter for anything?

>>43009939
>does it matter for anything?
Not for the end users.

>>43008837
All references are hit pieces. Either he bought into propaganda or is pushing it.
Shit, nigger, he's using a Forbes and a BBC piece to try to back his point.
I'd have believed him or discarded the point if he'd just said 'oh i'm afraid of my country censoring me' because there's countries out there actually doing that.
His other three reasons are actually valid (albeit he's being a pussy about it and closing shop instead of fighting on), which is why this one stands out as retarded.

>>43006299
Didn't Rome have desu already scraped, or I'm confusing it with EqD? It would be easier to parse links if the text of posts is already there.

>>43008991
i've downloaded all of those tonight, so there's a copy somewhere at least. i'll try to see if i can host it somewhere in the coming days.
i had to use proxies because pomf seems to ban you if you download too much at a time, and i don't exactly have time to go slowly.

>>43008997
All lain.la links from desuarchive:
https://u.pone.rs/pnrcyswk.tgz

>>43010754
Uh, I asked here: >>43006299

I'd insult you for using G5, but I have a feeling you're falseflagging.

>>43008525
>irc isn't gone, the web client is down.
Is Canternet still up? Last time I tried to connect to irc.canternet.org with another web client it errored out.

>>42998839
>https://infrablog.lain.la/pomf-announcement
>I had so many great ideas to fight this plague, like starting to inject stealer log malware into successfully cracked 7z files
What did he think to accomplish with this? Dox them? It's not like the FBI offices aren't public knowledge.

>>43010803
it is
>>43010496
thanks! downloading those too.

Twibooru derpi importer broke. It's been stuck for 11 hours already.

File: 1752268747241810.png (90.9 KB)

90.9 KB PNG

>>43009955
>flagrant gpt gibberish that has nothing to do with the discussion

File: ponkapresent.png (692.2 KB)

692.2 KB PNG

>>43011341
>NOO IT'S AI WAAAAH
Sorry, only natural posts here.

>>43011459
Zaid?

>>43011271
And the importer missed a lot of images. Again.
Seriously, the altboorus content is full of holes.

>>43011850
that's weird, because the way it failed was not actually failure of the importer, but merely failure of image processing. all the images would've still been sitting there waiting to be imported.

>>43011116
>it is
Looks like it. I'm in there now. Busy channel lol. I guess you weren't kidding.

>>43012037
Man, the entire server is deserted. We had this many users as the top channel does on our own private server way back. What do kids use to talk to each other nowadays when they're not using Discord?

>>43012082
Messengers, I guess. Telegram/Signal/Whatsapp/iMessages/etc.

Does Twibooru have a fuctional Tantabus importer?

>>43013042
Do any of the boorus actually have an automatic Tantabus importer? The "tantabus import" tag does seem to exist on all of them, but I think it's just added by the uploader considering there are only a handful of results.

>>43013602
I think AppleDash added Tantabus importer at some point.
[i]Edit[/i]: now that I re-read the archives, looks like he asked about it here: >>41713722 , but the decision was never made, so it was never added. All imports are manual.

>>43013791
Even though I use AI myself like any non-mindbroken individual, I'm conflicted on whether it should be automatically archived. It's much more important to archive the actual checkpoints and LoRAs than what can be made with it, and AI compositions (i.e. pictures that have some actual work put into it) are already allowed on Derpibooru, AFAIK. On the other hand, a quick search says there's about 30k images uploaded to Tantabus in one year, compared to about 200k for Derpi, so it might not even be that much of an issue (although I also see some images that are way older than 1 year in that list, so I'm not exactly sure how accurate that number really is).

>>43012817
>Telegram
is for groomers
>Signal
is for insurrectionists
>Whatsapp
is for jeets
>iMessages
is for itoddlers

File: dash shrug.jpg (242.2 KB)

242.2 KB JPG

>>43014258
You asked "What do kids use to talk to each other nowadays when they're not using Discord?" not "What are good alternatives to Discord?"

>>43014321
>>43014258 isn't me. I use Signal personally but that's just for people I know (along with Whatsapp for the tech-illiterate ones). I'd love if everyone still used IRC/Usenet, but that's obviously never going to be the case, so I'm just wondering where I can meet you guys outside of this board as long as it is NOT Discord (which I refuse to use).

>>43014258
>>Telegram
>is for groomers
and Russians ъyъ
>>Signal
>is for insurrectionists
and that's a bad thing because..?

>>43014517
Isn't it blocked in Russia? I thought only government-spyware "MAX" chat app that's directly tied to your digital ID was allowed.

>>43015036
Not everywhere yet.

>>43012032
Are you planning to somehow reimport the missing ones?

File: some derpi rules on ai.png (76.5 KB)

76.5 KB PNG

>>43014057
It's just that I thought Twibooru was a sort of aggregator site (I don't know if that's the correct term) that archives all the images from all the pony boorus, and to see it ignore Tantabus kinda bothers me.
>It's much more important to archive the actual checkpoints and LoRAs than what can be made with it,
I agree that those need to be archived as well, but why not the art that's genned with them as well, especially if we've got a dedicated booru for pony gens?
>and AI compositions (i.e. pictures that have some actual work put into it) are already allowed on Derpibooru, AFAIK.
They are, but AI-generated images that have little to no manual alterations made aren't allowed (see picrel, part of the description for the "ai generated" tag on Derpibooru); that's what Tantabus is for.
>On the other hand, a quick search says there's about 30k images uploaded to Tantabus in one year, compared to about 200k for Derpi, so it might not even be that much of an issue
Tantabus is still relatively new when compared to Derpi, so it'll be less active. Also, I don't understand how this comparison is a possible justification for Tantabus not being imported automatically. I don't mean to come off as whiny, it's just weird to me that Tantabus is somehow an exception when it comes to importing things to Twibooru.

>>43015036
Signal or Telegram? Telegram is functional while Signal is banned (but somehow werks on my machine)
also rhe insanity of RKN's bullshit just made everyone use VPNs lol

>>43015475
>but why not the art that's genned with them as well
Wouldn't it make more sense to archive seed, parameters and prompt? Pixels are just bloat. (Unless it's non-deterministic due to floating point numbers being shit.)

>>43015549
>Unless it's non-deterministic due to floating point numbers being shit.
NTA but that would be the problem for a fair chunk of them, yes.

>>43015549
>Wouldn't it make more sense to archive seed, parameters and prompt?
It would make sense, but a lot of the AI pics don't have the prompt, seed and parameters saved. There are some based prompters that post them in the descriptions, but they're in the minority. I think Tantabus strips most of the metadata from the files, though I'm not sure. And images and such that were sourced from sites like Twitter have that information stripped as well, unless I'm mistaken.

>>43015549
Unfortunately, different implementations of software used to generate and even different drivers influence the final output, so unless all that information is also included the final image may still be different from the settings that are used.

File: 1743270874220582.png (110.1 KB)

110.1 KB PNG

>>42998839
catbox always wins baybee

Alternatives for /mlp/ that are not NHNB or /poner/?

File: 69472.png (1.1 MB)

1.1 MB PNG

>>43015696
Tantabus doesn't strip PNG chunks with gen info, images that don't have them were uploaded without it
eg https://tantabus.ai/images/69472

>>43016744
It's worth noting that derpi DID strip gen info, and all forced-to-tantabus images from there have a high chance to not have it.

File: 2960695.png (1.4 MB)

1.4 MB PNG

>>43016747
Derpi stopped stripping info on images when they moved to philomena, long before AI
AI images on derpi had (and still have) PNG chunks, eg
https://derpibooru.org/images/2960695

>>43016744
What about JPEG files that contain it? Civitai files have generation data in the User Comment field, at least if the user has opted not to hide generation settings.

>>43010496
Just got done downloading this whole set besides one file at https://pomf2.lain.la/f/kn5wbp0w.7z which returns a 404 error (one other file was broken because the URL erroneously had a "4" appended at the end but removing that fixed it). It's about 20 GB total size. Is anybody here interested in me making a torrent of it (after I give it some rudimentary checks for pizza)?

>>43017175
Actually, I may wait with releasing it until uploads are disabled on Feb 14th and I'm sure nothing new has been submitted since >>43010496 was posted, but I'd still like to know if there is any interest first.

>>43017175
>torrent
Why not just make a host site serving the files with the same id? Because in the future nobody will seed the torrent, nor anyone will try to download the whole pack just because of a single file.

File: 1770454622139.png (387.4 KB)

387.4 KB PNG

>>43018277
Or do both, web server for quick access, torrent for bulk download and potentially setting up more mirrors. If you retain the directory structure or use hardlinks, you could run both off one copy of data easily, no?

>>43018298
Would the torrent include just the files or the entire site hosting them?

>>43018277
>Why not just make a host site serving the files with the same id?
Don't have the capacity to provide something like that right now, I'm afraid.

>Because in the future nobody will seed the torrent
Won't happen because I'll keep seeding the torrent well until anybody has anymore interest in it (I'm still seeding Ponibooru-All-Unrated too), plus submitting it to archive.org as well.

>nor anyone will try to download the whole pack just because of a single file.
About that: compressing the entire archive to a single file saves a not-unsubstantial amount (2 GB). However, it's obviously not as convenient as having each file available individually (or at least as big as the minimum block size of the torrent would be, which is probably 4 MB for an archive like this). The only real uncompressed files I see though are a bunch of Safetensors files, WAV files and a bunch of text files with a total size under 200 kB so they're not an issue (oh and a couple of TAR files that I'll have to try and see if they would benefit from compression or not). Are there any concerns with me just making XZ archives of these files while keeping the rest uncompressed (besides their original compression)? That's what I'm planning to go with for now, so if anybody objects with a good reason let me know beforehand.

File: derpyhooves.png (173.2 KB)

173.2 KB PNG

>>43018612
>Don't have the capacity to provide something like that right now, I'm afraid.
Fuck it, I'll just ask: what does it take to host services or do other tech things for /mlp/? Some further questions:
>do I need to be behind 7 proxies, 100% anomalous man like Floorb, or can I be more like Corp/Wolvan?
>how stressful is it?
>how to not get v& from someone abusing user generated content?
>what threat model should be assumed?
These are known unknowns I haven't yet encountered doing tech elsewhere. Good faith responses, ideally grounded in experience, would be appreciated. I know answers depend on specifics of service or project, but I'd like to gauge the general sentiment for now.
I just want to use my brand of autism to poniponi, man.

Does anyone know what the biggest repository of greentext is? Or of any not listed in the OP? When I read greens from pastebin/ponebin I often find links to greens/their authors that aren't on ponepaste; it's like a labrynth.
Also does anyone have the screencap of /mlp/ doing a pony-themed bit in the style of Chris Hansen's To Catch A Predator? I looked for it on yerf (from the OP) but no dice.

>>43018869
As the guy you responded to, I have the same questions as you do, hopefully an anon with some experience can elucidate. I already have some answers myself but I'm in no way knowledgeable in this field.

Anyway, as an FYI to everyone, I'm going to get everything from >>43008991 as well, just wasn't sure if I had the disk space required right now, but it seems to be mostly images and some MP3s so I'll get it when I'm done with this archive. Regarding hosting, since Filly said he got everything downloaded too, maybe he's willing to host it somewhere? Then I can add the location as a web seed to the torrent file before uploading it to archive.org. Obviously we'd need to sync the XZ files I'm making beforehand.

>>43018869
>I just want to use my brand of autism to poniponi, man.
Btw, I definitely echo that sentiment. I just want to share ponies with others and I have to worry about such bullshit like literal three-letter agencies trying to ruin your site just because you allow free speech. Fucking ridiculous, go fight actual criminals.

>>43018895
>greentext
The most likely contenders for "biggest green archive" are those who save entire threads, such as archived.moe.
>groomtext
Best I can do is shill:
https://www.fimfiction.net/story/579608/to-catch-a-couturier

>>43015475
Unfortunately, it seems that at least for now Booru owners are siding with the artists on this one so we'll need to backup Tantabus images by ourselves. I don't think even Rome backs it up. I guess if you meme hard enough you can actually sway popular opinion. It is what it is.

>>43019265
There's already too much AI shit showing up on the boorus. We're having AI be shoved into everywhere else it doesn't belong, we don't need it shoved into the boorus too. And it's not like there's anything of real value to any particular AI gen either, because you can always make another gen in an instant.

aaaand here comes the luddite to throw a fit in the face of already mentioned facts.

rangeban india

>>43018974
i have a plan to host direct links to each of the individual files in the works
if you'd like me to host an archive of them all, do let me know and i'll be happy to do so!

>>43019293
Nice opinions.

>>43018612
You can upload everything here and thrn tell me when you're done so I can make it accessible
https://iwiftp.yerf.org/Shared%20by%20other%20people/Upload%20form/
You won't see files on it because someone upload p t h c so I disabled viewing
>>43018895
Here are a few
https://iwiftp.yerf.org/Pony/Fan-fiction/
Some others might be on the mulpwiki backup I finally fixed
https://mulpwiki.batpony.party/index.php/Main_Page

>>43019556
Thanks, I'll get back to you when I'm done with this archive.

>>43019582
Okay, I'll upload everything from the Desu archive there first. I have only done very quick checks for pizza (I didn't see anything but I haven't combed through the data or anything), so make sure to look it over by yourself too.

>>43019629
Here's the link for now, if you add files they'll show up.
https://iwiftp.yerf.org/share/pomf-desu/

>>43019803
All files for >>43010496 have been uploaded (with the WAV and Safetensors files compressed with XZ). I got a couple of times the connection seemed to error out on my end and then recovered, but it doesn't say that any files failed so I assume it all came through. I'm at about 20% done with downloading >>43008991 so that one is going to take a bit longer. I'll create the torrents after Feb 14th when uploads shut down.

>>43020477
Getting there again.

File: 1760755625456204.png (77.5 KB)

77.5 KB PNG

what the fuck is this ad?

>>43021830
>wronghole.xpm
>minecraft island temple
>https://ponerpics.org/images/7353935
Any questions?

>>43021830
Ads? What's that?

>>43021952
something I have on because some are funny

>>43018869
I don't have much experience, but I'll give my 2 cts. I think when it comes to hosting a service, you really should Just Fucking Do It. As with anything else in life, you don't learn anything if you don't actually start doing it yourself. I think the main threats for a webserver are:

1) You get hacked somehow and the data of the server gets wiped
- Fixed with automatic backups on a local server (though you have to secure the local server especially so nobody malicious can get remote access)
2) CP is uploaded
- Can be fixed by vetting uploads and/or full-disk encryption so that in the case you're too late to delete something and feds bust down your door they at least don't have access to the data, so nothing to charge you with (which they don't have in the first place since you're not complicit, but they could charge you with negligence)

The DDoS stuff is not as bad in comparison, since it'll just mean your server is offline for users. Annoying, but that's about it, nothing really dangerous. A far larger problem is terminally online losers with an axe to grind trying to dox you. Make sure nothing links to your real identity, at the very least. Don't register a domain with your real name OR common nickname or email address. Use KYC-less crypto to pay for a host, and in general practice reasonable OPSEC. But even then, look at the cases in this thread where things went wrong: several pony hosts had to shut down because things got out of hand for them, but their lives haven't been ruined any more than they already were as far as I know. You may start a service and have to shut it down a year down the line, but the chances of it impacting your real life in a negative way are probably not that big.

One thing on OPSEC though: don't trust users in this community to keep your identity secret. If you use a different nickname when you're administering your server, but your IP address is known and you post "anonymously" on a site like Derpi (or any other site in this community), don't be surprised when they suddenly "leak" this information in a private chat. These are not companies you're dealing with, but autistic shut-ins and some have VERY poor principles.

Just noticed that the OTHER Equibooru (that collected art and not photos) went down.

>>43022619
>Equibooru
What's that?

>>43022664
It was a booru aggregator focused around equine art. It pulled from e621 and IB, and had a dedicated MLP section.

>>43022669
Just art, no zoo or anything?

>>43022742
Just art. Like I said, it's the OTHER site called Equibooru. Had a few mirrors like .eu that don't seem to be up.

An amusing possibility is that it got taken down because the authorities thought it was that other site though.

>>43022758
.eu sites are a bad idea to begin with. The EU is pretty much the most corrupt Western organization in existence and I don't see it beyond them to do something as stupid as you described. Incompetent at best, malicious at worst.

>>43021830
A shitpost, I assume.

>>43019803
Here are the hashes for the uploaded files from the Desu archive so you can check to see if they match: https://litter.catbox.moe/mppivc.md5 (make sure the XZ copies are uncompressed first before you check them). They were crosschecked with Filly's copies, so they should be correct. Once he has checked the Cytube copies that he has to see if they match mine I'll upload those to the server too (unless you've already got them, in that case please let me know).

https://ponepaste.org/dumps/
Has anyone noticed that the newer dumps got broken at some point? The current dump is completely empty and the deltas for some of the more recent ones look suspiciously small as well.

I have been mirroring these automatically so I don't know when it started but looking at the file sizes it seems like they have been empty since May 2025 at least.

Maybe I should use something else to archive ponepaste but I really don't have the time to build a scraper right now. I was kind of relying on the dumps "just working".
Don't suppose anyone else has a better ponepaste archive?

Floorb pls fix.

>>43024613
No just upload them

>>43024815
I'm uploading the Desu archive part to Archive.org right now, it should be done tomorrow (it's really slow). I'll upload the Cytube part to your server when it's done.

>>43006756
I think the best way to do this would be to just take the Flash files from the leaks, crop out the head from the side view and then convert it to SVG in Inkscape (and tweak it a bit there). I could probably look into that if you're okay with me taking this approach. Just so you know I'm not going to be drawing anything (because I don't have the skillz).

>>43024789
i have looked into this a few times and so far not figured out what the issue is. i can look again later today / tomorrow.

>>43025108
While I appreciate the offer, I think it's probably best not to involve the leaks. I'm sure it probably doesn't really matter, but I think I'd be more comfortable if it were an original composition so to say. But another thing that people could do that doesn't directly involve drawing: suggestions for character themed backgrounds. What would be in the background image for a Pinkie themed background, etc.

>>43025164
this time, I figured out the issue right away. server was running out of RAM each dump, and producing an empty one after the dump process died. running a dump now that should be up to date.

Do we already have a contingency plan for when catbox.moe inevitably goes offline?

>>43025191
I don't think so, no.

>>43025333
If a kind anon like >>43010496 can get me a list just like that one but for https://files.catbox.moe/ links* I'll at least try and see if I can download everything and we can worry about hosting later (hopefully it's not TBs of data). And of course everyone else is free to make their own backups in addition to mine.

* There's another URL with links on Desuarchive - https://de.catbox.moe/ - but the certificate is invalid and Catbox has HSTS turned on, so it can't be reached easily. The other URL, https://sg.catbox.moe/, isn't on the archive. Oh and of course there's Litterbox but I'm not going to bother with those myself (feel free to do so if you want to though).

>>43024815
All files from the Cytube part have just been uploaded (the Archive.org web upload errored out on me so I thought I'd do this one first and do the Archive.org upload again later using the command line). There were numerous intermittent connection problems on my end, and 1 file did actually say it failed, but it still finished saying all files were uploaded successfully, so I don't know. Run md5sum to see if there are any problems or not (though I haven't crosschecked this set with Filly's yet).

File: full.png (264.9 KB)

264.9 KB PNG

Anybody here good with gallery-dl?
I'm poking at it and I've figured out that it's basically impossible to properly dedupe or avoid downloading excessive amounts of data when scraping multiple Philomena sites because that Elixer shit fucks with every image posted to it.

Anybody know a way to pass multiple post processor values into the sqlite3 database as multiple rows? it would make deduplication somewhat possible at least to have both the new and original hash to check against.

>>43018869
For achiving purposes you don't have to worry, the real illegal stuff is already deleted by the time you archive it. (Of course, if your country bans fictional art, then move to a free country or something, I dunno.) ToC is the biggest problem, majority of hosts ban completely legal content. Copyright is another problem, but this mainly becomes a problem when you host new leaks.

>>43025175
Thanks for the new dumps. Is this a permanent fix or is it going to recur as ponepaste keeps growing? Seems kind of strange for it to consume more memory for what should be a streaming operation.

>>43025438
> because that Elixer shit fucks with every image posted to it
"fucks" in what manner? If you mean the "image optimizations", not sure if those are due to the software or just because each admin configures it differently. But if it was part of the booru software I would expect it to give the same result on all instances.

> multiple post processor values into the sqlite3 database as multiple rows
Multiple rows are not possible, but might be planned for v2
https://github.com/mikf/gallery-dl/issues/5006

You can override the archive key though. Not sure if that helps your use case. But if you want to dedupe according to the "original" hash just store that instead of the image key?
https://gdl-org.github.io/docs/configuration.html#extractor-archive-format

Alternatively you can just download the images to a tmp dir and use a postprocessor to move or delete them according to your own custom logic. I use that to handle cross-account reblogs on sites like mastodon.

>>43025418
https://iwiftp.yerf.org/share/lain_la-cytube-mlp/
Also unrelated but here's the files from smutty.horse (spineless owner)
https://iwiftp.yerf.org/share/u_smutty_horse/
Download them because I will delete them in the future

>>43025703
Sounds to me like you're the spineless owner if you're too afraid to keep them archived.

>>43025857
Meanwhile they'll always be up and archived at u.smutty.horse

>>43026015
based

>>43025703
> Download them because I will delete them in the future

How do I do that? I tried configuring rclone for the webdav URL and that worked fine for all the normal folders inside archvie/. But it doesn't see the archive/share/ folder that those links under.

I assume thats because the share folder itself is normally hidden and only specific subfolders are accessible by URL. But even trying to list it via a full path like so:
> rclone lsd yerf:archive/share/u_smutty_horse
> rclone lsd yerf:share/u_smutty_horse
doesn't work.

Having to manually select and download each file via the web UI would be kind of soul crushing. And probably induce carpal tunnel.

>>43026184
I forgot about that, share is a virtual folder. But in web favorites it should be accesiable under /Shared by other people/Upload form

>>43026249
Webdav*

>>43025579
>You can override the archive key though.
yes, you can do this to store the image hash, philomena API json provides both the new hash after the site screws with the file as well as the original hash of the file.

The only way to avoid triple scraping the same file from multiple sites that import from each other is to store BOTH hashes for each file and even then it's not a sure thing.

This is why "optimizing" files is fucking retarded.

>>43026280
Wow! Two whole hashes! It really does suck that computers can't handle two whole equality operations in 2026. I hope the technology arrives soon.

>>43026249
Thanks, found it. I'll start mirroring tomorrow.

>>43026280
> The only way to avoid triple scraping the same file from multiple sites
No, I don't think your problem is solveable with gallery-dls current archive mechanism. It is very limited.

You could always fork the project and implement a custom archive, which is honestly not as bad as it sounds. I also forked it to do some custom fixes for a few websites and some added functionality (like improved skips).
Or change job.py to check the exit code of the postprocessors. Then the exec PP could act as a sort-of plugin mechanism.

>>43026412
Actually, looking at the docs it *seems* like its doable using the actions mechanism. Please bear in mind that it's extremely new (4 days old) so I have not used this myself. I'm just going off what the docs tell me.
https://github.com/mikf/gallery-dl/discussions/8960

Based on my interpretation you can do something like this:

Define an exec postprocessor that runs at some stage prior to an image download. That postprocessor can then do its own duplicate matching with any complexity you desire.
If the file should be skipped then the postprocessor should send a (non-fatal) signal to the downloader process, like SIGUSR1.

Then define a signal action for that signal that sets a flag in gallery-dl to skip the image/download.
https://gdl-org.github.io/docs/configuration.html#signals-actions

The current image/post should now be skipped.

Now I'm not entirely sure if those flags are reset for every new post and the docs don't tell me, but the discussion kind of implies that it does. I dunno, test it out I guess.

>>43025191
Get the sharty taken down and you'll remove the #1 threat to catbox.

>>43019129
>>43019582
Thank you, gentlemen.

File: 1765866400179952.png (390.4 KB)

390.4 KB PNG

>>43026347
you are a retard, thanks for nothing.
>>43026473
this is actually potentially very helpful, thank you.
I've considered just running another instance of gallery-dl after the download and just using --no-download with a different config file that pulls the alternate hashesand instead and loads them into the DB using POST instead of AFTER, but the whole point is to try and not be that jackass who is making 500 API calls a second just to compensate for a program being retarded and needing to pull the json 5 times per image.
Shit programming grinds my gears.
I'll see if your solution works, it's the best lead i have right now, thanks.