Mordhau
 Kaervan
  • Likes received 10
  • Date joined 1 May
  • Last seen 11 May

Private Message

21 10
  • 1
  • 11 May
 Kaervan

@Runagate said:
The offline bots have the exact same stats as you do

I don't know about offline bots, but this is not the case for Horde bots. Horde bots do not have an attack reset duration and have unlimited stamina. This is why Horde matches lead to hiding at the top of ladders shooting arrows at the bosses. It's obvious when seen side by side:

21 10
  • 8 May
 Kaervan

It would be fantastic if the AI for bots didn't have a much faster attack speed than players. They already have nearly perfect parry and block, they don't need to attack twice as fast as players.

21 10
  • 4 May
 Kaervan

@Fassan said:

@Kaervan said:

@Fassan said:

@ExCTyrdlu said:
@Fassan
My 22 player server running on 1x 2.095GHz, 2GB RAM, 20GB SSD 100/100 Mbps, tickrate is 60. I haven't problem with overload/lags on full server.

This is my cpu load on 16 player:

But I am running at 120 tickrate, is that overkill?
When I was running 32 players sometimes I got small lags and the cpu then spiked for like a minute.

A dev confirmed in another thread that the server only requires one core. The official servers are quad core ~4ghz systems running one server per core. The deciding factors for a good experience are

CPU speed - player count dependent on core speed
Bandwidth - need sufficient throughput for player count

Your tickrate will have a direct impact on both of these. Your server is receiving data from all the clients for what they are doing, and sending the data for what everyone is doing to everyone. The tickrate is the rate that the client updates are received and sent. A higher tickrate means that the client will send that many updates to the server per second, and your server will send those updates to clients. This will drive more cpu load for processing and more bandwidth usage (PPS, BPS).

To compare, I'm running a 32 player server on a 2.1ghz CPU at a tickrate of 60, and haven't heard any complaints. Here's a clip of my server load while it was full yesterday:

Thanks alot for the detailed information :).

I have plenty of bandwidth and am running on xeon 2.5 ghz.
I am now running on 90 tickrate just to see if the intermittent lags are gone.
It worked very good before on 120 tickrate but once in a while I had lags spikes and the cpu was at 98.5 % for a few mins.

Are you running virtual or physical?

Have you tested 60 vs 90 vs 120 tickrate in terms of the player experience?

Trying to find the sweet spot for player experience vs server load.

I haven't done any testing above 60 tickrate. With the increased packets per second, I probably wouldn't go over 60 for an internet server because all the routing devices between the player's computer and my server are potential points of failure, where failure here is anything from dropping packets to causing higher latency due to congestion. If I I was in a country where gigabit fiber internet was the standard, I would probably go to 90 or 120. Also, LAN servers should have no problem with higher tickrates provided the server has enough compute to process the information

21 10
  • 4 May
 Kaervan

What CPU are you running on, and what is the bandwidth to the server?

21 10
  • 4 May
 Kaervan

Rahlekk, that looks like there's already a process bound to the port. Check to make sure the previous process has exited.

21 10
  • 4 May
 Kaervan

I posted in reply to the announcement, but that was probably the wrong place. Here's what I've noticed:

There's a long delay after LogInit: Finish loading of assets took 0.000015 seconds, and the server eventually gets to the point where it runs and cycles maps, but there's never a selection for announce/browser server. Previously the log would show entries where it polled na01, na02, eu01, eu02, ap01, and ap02 to determine where it should list, but those log entries are absent.

A log sample can be found at https://pastebin.com/b3CXmCai

21 10
  • 4 May
 Kaervan

As a career systems administrator, I have some recommendations for your notes:

  • I would highly recommend installing/setting up sudo for your regular user to be able to install packages, set up systemd, and do other administration things.

  • the useradd command was not mistakenly used in the original post for ubuntu. The steam user does not need a password, and giving it one means a user can log in as it. sudo -iu steam can be used to switch to the steam user after setting up sudo for your regular user.

  • separate into sections the things that are system administration like yum and firewalld in a way that show which access level should be used for each.

Feel free to DM me on discord if you'd like any further explanation on these points or for any other linux admin and security questions you might have. I use the same name.

21 10
  • 4 May
 Kaervan

With this server update, we have people in discord who are unable to get their servers listed in the server browser. There's a long delay after LogInit: Finish loading of assets took 0.000015 seconds and the server eventually gets to the point where it runs and cycles maps, but there's never a selection for announce/browser server. Previously the log would show entries where it polled na01, na02, eu01, eu02, ap01, and ap02 to determine where it should list, but those log entries are absent.

21 10
 Kaervan

@Fassan said:

@ExCTyrdlu said:
@Fassan
My 22 player server running on 1x 2.095GHz, 2GB RAM, 20GB SSD 100/100 Mbps, tickrate is 60. I haven't problem with overload/lags on full server.

bandwidth.PNG

This is my cpu load on 16 player:

mordhau2.PNG

But I am running at 120 tickrate, is that overkill?
When I was running 32 players sometimes I got small lags and the cpu then spiked for like a minute.

A dev confirmed in another thread that the server only requires one core. The official servers are quad core ~4ghz systems running one server per core. The deciding factors for a good experience are

CPU speed - player count dependent on core speed
Bandwidth - need sufficient throughput for player count

Your tickrate will have a direct impact on both of these. Your server is receiving data from all the clients for what they are doing, and sending the data for what everyone is doing to everyone. The tickrate is the rate that the client updates are received and sent. A higher tickrate means that the client will send that many updates to the server per second, and your server will send those updates to clients. This will drive more cpu load for processing and more bandwidth usage (PPS, BPS).

To compare, I'm running a 32 player server on a 2.1ghz CPU at a tickrate of 60, and haven't heard any complaints. Here's a clip of my server load while it was full yesterday:
mordhau.png

21 10
  • 3 May
 Kaervan

oooh, datadog. well aren't you fancy. I was going to post my grafana snapshots for full 32 player server, but it looks pretty much identical to your graphs. It looks like you could easily go to 28-32 players based on your image if you wanted.

21 10
  • 3 May
 Kaervan

@Peacerer said:

@marox said:
You only need a single core for the server, we run 4x64p on E3-1270 v6 machines.

No wonder it runs like crap.

I don't understand how you're under the impression that the dedicated server runs like crap. The server is unreal engine, which is long known for only needing one core and the speed of the core being mostly the only thing affecting how many players it can support. If your client experience is poor, you should look into things that are more in your control. Rendering and compute performances are reliant on your system specs and graphics settings. network performance and latency are dependent on the distance to the server you play on, the load on your router, your connection to your router (wifi vs wired, and any switch load in between), as well as the available bandwidth for your internet connection.

Maybe to address the issues you see, you should start a post about it seeking assistance rather than flaming a likely unrelated thread.

21 10
  • 3 May
 Kaervan

Sorry for not understanding your earlier posts. You had already answered some of my questions.

That's wild. I just read through your pastebin file again but slowly this time. There's really no indication there that anything is wrong. The first part of the address in your post makes me think the server is in Brazil, and the log shows the server selecting eu01 as the announce server. There are only NA, EU, and AP endpoints but there is a South America selection on the matchmaking screen in game. I'm wondering if the selection there isn't the best and it's not being listed because of some geoip issue.
With all you've said and done, maybe you need to email [email protected] and ask if they could look for your address in their logs to make sure it's not something on their end.

21 10
 Kaervan

Thanks for the confirmation marox. There are threads here and there asking about running multiple servers on one host. I had suggested they try changing the queryport, but they say it doesn't work. For the official servers, are you using different ports on the same address, or assigning VIPs to an interface and binding the additional servers to those addresses somehow?

21 10
  • 3 May
 Kaervan

Hi!

I'm running a 32 player server, and am not going higher because of the usage I'm seeing and the system I'm running on. Hopefully some details on my setup can help you out. I'm about to set telegraf up and push server stats into a grafana instance, so I can share that later when I have some numbers too. If anyone has some rough numbers on running a 64 player, would be interested in hearing.

I'm running in a VM on a VMWare ESXi system, the specs for the VM are as follows:

2 vCPUs (Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz) with no overcommit, so they have dedicated cores.
4GB Mem
40GB HD (virtual disk is on a samsung PCIE m.2 drive - super fast)
This is running at my house where my internet connection is 1gbit down and 45mbit up

CPU usage

Originally, I was running 4 CPUs, but the server was barely using 3 of the cores. Reducing to 2 cores, players report no difference in performance, even running 60 tickrate. When the server is full and people are murdering, one core will run up 65-75% of one core, and the other core will bounce around between 10 and 30%.

Memory Usage
I deployed from a template that hd 16GB memory assigned to it, but the VM was only using ~1.2GB, so I reduced to 4gb to leave some room for file caching. At 4gb, the server VM has not used any swap space.

Storage
The m.2 drive is super overkill. Server map changes finish in ~2 seconds tops. A regular sata SSD would likely see similar performance.

Network usage
The absolute highest I ever saw the server get for 32 players was about 6.5mbits upstream and 8mbits downstream.

Afterthoughts

I toyed with building a cheap intel i5-9600 system to get to 64 players because I'm pretty sure I'm CPU bound to around 40 players on my 2.1Ghz xeon. The 9600 runs up 4-ish Ghz, and the system would cost ~600 to build from nothing.

For those player counts, some ballpark specs that should do what you want would be:

16 Players

CPU: 2 or more cores @ 1.8ghz or higher
Mem: 2GB
Disk: Doesn't matter, whatever has capacity
Bandwidth: 5mbit down, 3 mbit up

32 Players

CPU: 2 or more cores @ 2ghz or higher
Mem: 3GB
Disk: Doesn't matter
Bandwidth: 8bmit down, 7mbit up

64 players

CPU: 2 or more cores @ 4ghz or higher (fairly confident)
Mem: 4GB (Guessing here)
Disk: doesn't matter
Bandwidth: 15mbit down, 10mbit up (maybe +2-3 for each)

21 10
  • 2 May
 Kaervan

@Destroptor said:
@Kaervan

Thank you so much for the detailed explanation. I found the problem elsewhere though. I might have derped and installed in a directory where root access was necessary. After moving the steam folder to my home folder, and a chown command it would then create the folders and files needed

Good to hear you got it worked out!

21 10
  • 1 May
 Kaervan

What Shifty meant was to look in a game client that is on the same network as this server. You should be able to see the server on the LAN tab in the server browser. If it is not there, The system it is running on is not allowing the packets it needs, whether it's inbound or outbound. If it IS in the LAN section, what does your network setup look like?

Is this a 1:1 NAT?
Are there any ACLs on the switch port where this server is?
What are you using to manage your firewall, iptables, firewall-cmd, ufw?

what does your iptables-save output look like - AFTER removing sensitive information like IPs.

21 10
  • 1 May
 Kaervan

@PogoPuschel said:
A short netstat shows that the game uses port 27015 and 7777.
So use also need to change that i think.

[URL]
Port=7777

good call!

21 10
  • 1 May
 Kaervan

The game and server use the unreal engine. Checking the unreal wiki, it looks like you're looking to change the queryport. A quick grep of the server directory shows this;

[email protected]:~$ grep -ir GameServerQueryPort *
mordhau/Engine/Config/BaseEngine.ini:GameServerQueryPort=27015

So the core configuration file has the steam port. You want to change this, but that file is not where you want to do it. Alongside the Game.ini file is an Engine.ini file. The Engine.ini file in mordhau/Mordhau/Saved/Config/(Linux|Windows)Server/ should take precedence for any entries there and use the defaults from the BaseEngine.ini file. The relevant parts you want in Engine.ini are:

[OnlineSubsystemSteam]
GameServerQueryPort=27015

Where you change the port to what you want. I have not tested this myself, so if it doesn't work, changing it in mordhau/Engine/Config/BaseEngine.ini should technically do what you want.

21 10
 Kaervan

@Destroptor

To get to the point where I ran into the Saved directory missing, I followed the original post to that point. My directories looked like this:

[email protected]:~$ tree -L 1

├── linux32
├── linux64
├── mordhau
├── package
├── public
├── siteserverui
├── Steam
├── steamcmd.sh
├── update_mordhau.txt
└── update.sh (my script for updating the server with steamcmd)
7 directories, 3 files

[email protected]:~$ cd mordhau/
[email protected]:~/mordhau$ tree -L 1

├── Engine
├── Manifest_DebugFiles_Linux.txt
├── Manifest_NonUFSFiles_Linux.txt
├── Manifest_UFSFiles_Linux.txt
├── Mordhau
├── MordhauServer.sh
└── steamapps

3 directories, 4 files

[email protected]:~/mordhau$ cd Mordhau/
[email protected]:~/mordhau/Mordhau$ tree -L 1

├── AssetRegistry.bin
├── Binaries
├── Config
├── Content
├── Intermediate
├── Mordhau.uproject
├── Plugins

6 directories, 2 files

The OP says I should be expecting the directory named Saved in that last command output, but it wasn't there. After some digging around, I ran the MardhauServer.sh script in the second directory listing above with just sh MordhauServer.sh. It did a bunch of stuff on startup and ran in the forground. After waiting a bit, I hit CTRL+C and looked in /home/steam/mordhau/Mordhau and the Saved directory was there with Game.ini and others inside of it.

If this doesn't help, I'll be home around 6:30PM PDT and can jump on discord to give you a hand.