Tio

tio@social.trom.tf

A bunch of atoms that are curious about the world, but furious about the society. We should help all humans to have a decent life without any strings attached, so that we can understand us, the atoms, and the rest of the universe. And create a smarter and saner society.

Since 2005 I make documentaries, write books, and enable many trade-free platforms for everyone.

Network posts

Earth

@tio:matrix.org

https://www.tiotrom.com/

Tio

4 years ago

Tio
4 years ago

!Friendica Admins

idk what I have done to my Friendica but it shows a 502 Bad Gateway nginx error quite often....I added a few RSS sources from Reddit and that seemed to have messed it up but I don't understand what. I removed them since...

The nginx error logs show stuff like 2021/08/02 07:05:57 [error] 2732#2732: *1092 upstream timed out (110: Connection timed out) while reading response header from upstream, client: ******, server: social.trom.tf, request: "GET /ping?format=json&uid=22&_=1627677955767 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php/php7.3-fpm-friendica.sock", host: "social.trom.tf", referrer: "https://social.trom.tf/network"

Didn't sleep trying to figure it out so am tired sorry for the messy message.

Basically now loads fine for a few minutes, then loads very slow to the point of nginx giving up as explained above. If any of you has any idea what I should try let me know. Gonna have some sleep now, not good to work on such things when you are zombie tired, you may mess things up more.

Thanks!

!Friendica Admins

Friendica Admins reshared this.

in reply to Tio

Tobias

in reply to Tio 4 years ago

Not using nginx myself, the StackOverflow answer seems to be to increase the proxy wait time. Question is why Friendica needs more time to answer.

Is there any hint in the PHP logs or the logs from Friendica?

in reply to Tobias

Tio

in reply to Tobias 4 years ago

@Tobias Yes if I add fastcgi_read_timeout 300; proxy_read_timeout 300; to the nginx.conf seems to work, but it takes a while to load...

Question is why Friendica needs more time to answer.
That's what confuses me.... it happened right after I added 10 or so RSS feeds as my contacts. I deleted them as quick as I realized that but it seems it made no difference. Something broke.
Is there any hint in the PHP logs or the logs from Friendica?
I do not know what to look for in logs more specifically. There are lots of them drive.tromsite.com/s/wgEzn8YG8…

@Tobias

Unknown parent

Tio

Unknown parent 4 years ago

@Michael Vogel i.imgur.com/kU0USu5.png There are a few processes already opened by Friendica. If you can take a look - can't even upload an image here. Also, is it normal to take many hours to restore a Friendica database of around 6GB?

@Michael 🇺🇦

in reply to Tio

Michael 🇺🇦

in reply to Tio 4 years ago

Have a look at the php-fpm processes. Possibly you are running out of them.

in reply to Michael 🇺🇦

Tio

in reply to Michael 🇺🇦 4 years ago

@Michael Vogel Could you give me more info please?

@Michael 🇺🇦

Unknown parent

Tio

Unknown parent 4 years ago

@JB Carroll pm.max_children = 5 and pm.start_servers = 2 if that's what you are asking. I increased these two now to see if it helps. And yes if I restart the php service it is temporary fixed for a few minutes. So maybe that's the issue?

@JB Carroll

Unknown parent

Tio

Unknown parent 4 years ago

@JB Carroll Increased to 50 so lets see. Is there any I have to increase besides pm.max_children = ?

@JB Carroll

in reply to Tio

Michael 🇺🇦

in reply to Tio 4 years ago

max_children of 5 looks really much too few. You should have a logfile for this where the system should complain about too few children.

in reply to Michael 🇺🇦

Tio

in reply to Michael 🇺🇦 4 years ago

@Michael Vogel @JB Carroll Increased to 50 and seems to work a lot better now. I need to test for the next hours. But if that fixes it I'd be over the moon happy. Thank you and I'll update here!

@Michael 🇺🇦 @JB Carroll

in reply to Michael 🇺🇦

Tio

in reply to Michael 🇺🇦 4 years ago

@Michael Vogel @JB Carroll How many pm.max.children would you recommend btw?

@Michael 🇺🇦 @JB Carroll

Unknown parent

Tio

Unknown parent 4 years ago

@JB Carroll 50 seemed too low, we reached that in 10 minutes. Increased to 500. Interesting that until last night we didn't experience an issue with this. Now I see at time over 50 " php-fpm: pool friendica" processes at once. Would that be normal?

@JB Carroll

in reply to Tio

Michael 🇺🇦

in reply to Tio 4 years ago

There are formulas out there that calculate the best value depending on your memory size, free memory and so on. The best is to have a look at the logfiles to possibly increase the value when havwe got problems.

Tio likes this.

in reply to Michael 🇺🇦

Tobias

in reply to Michael 🇺🇦 4 years ago

@Michael Vogel @JB Carroll @Tio if you find a good value, maybe you can add a comment in the nginx example config file about it? Not as fixed value, but as reminder for the admin that this value needs to be looked at.

@Michael 🇺🇦 @Tio @JB Carroll

Tio likes this.

in reply to Tio

Tio

in reply to Tio 4 years ago

So with pm.max_children = 500 pm.max_requests = 1000 everything seems to run smoothly. So far so good. Thanks a lot people! You are always so friendly and helpful.

in reply to Tio

muppeth

in reply to Tio 4 years ago

Personally I am not really fan of setting max_children to cosmic values. Increasing the number is IMO dead end. The moment you have all children processes busy your load average will be too high and its the cpu that will become your bottleneck, and once you reach that there will be no extra value to kick up. Surely distro defaults are low, but setting to 500 seems also crazy high.

@Tio check gist.github.com/holmberd/44fa5…

I noticed that for things like social networks where amount of users online is one thing but the fact there is always content being pulled in and sent out so your instance is busy constantly even without anyone online, setting pm to static is better approach.

Adjusting child processes for PHP-FPM (Nginx)

Adjusting child processes for PHP-FPM (Nginx). GitHub Gist: instantly share code, notes, and snippets.

^Gist

@Tio

Tio likes this.

in reply to muppeth

Tio

in reply to muppeth 4 years ago

@muppeth Yes I've heard that before, that is recommended to setup to static. I should try that and see how it goes. Thanks a lot!

@muppeth

in reply to Tio

muppeth

in reply to Tio 4 years ago

The good thing about it is that you dont have to worry about start_servers, spare ones and all that stuff :) . With friendica (or any social network but those originating from Zotlabs specially) works all the time so better to keep it static IMO as your php is busy at all times.

in reply to muppeth

Tio

in reply to muppeth 4 years ago

@muppeth I'm just worried I may destroy it haha. I will have to look carefully into how to properly setup the static value. I am a bit in the dark when it comes to calculating the max_children for friendica. But I'll need to look into that for sure. I am wondering tho, even with that cosmic number pm.max_children = 500 if our resources seem to be really ok, is that a bad thing? Now our Friendica works really well and it barely uses our CPU or RAM.

@muppeth

Unknown parent

Tio

Unknown parent 4 years ago

@Adam @muppeth True. I went down again to 50. Maybe 50 is not that insane.

@Adam @muppeth

⇧

Tio 4 years ago •

Tio
4 years ago