Page Index Toggle Pages: 1 Send TopicPrint
Normal Topic DDO and Lag: A History Lesson (Read 2896 times)
Rose-tinted Goggles
Waterworks Kobold
**
Offline


I Love Nostalgia!

Posts: 191
Joined: Jun 3rd, 2018
DDO and Lag: A History Lesson
Jan 9th, 2020 at 1:25am
Print Post  
I did some digging into the lag issues plaguing the 2010/2011 time frame. Where lag was arguably the worst it had ever been in DDO for raids. It turns out I may have been wrong assuming that due to this being the period of time where DDO was at its most popular, it put massive strain on the servers, causing the unplayable lag.


"DPS Lag" as it was called back then made raids nearly unplayable during heavy combat scenarios (boss fights). Around 2009/2010, players (and Devs) began assuming that TWF is the cause of the "DPS Lag" due to the increased number of attacks and procs going off.


The lag got so bad in Shroud that the the players put together a forum post to recruit TWF players for testing lag in Shroud. Some developers joined in to monitor the test.


A week later the Dev Eladrin made a post explaining how after doing internal testing with TWF and reducing off-hand proc chance by 45%, saw significant improvments in lag. Planning on reducing off-hand proc chance by 20% for live servers. As youll see in the post below, all this did was delay hitting the bandwidth cap by slightly lowering the total bandwidth limit each TWF player could reach.


A day later, the legend himself, Vanshilar made a post detailing his finds on how weapons with damage effects increase server bandwidth more so than weapons without. On the second page of this thread, Vanshilar points out the bandwidth cap of 20500 bytes and upon reaching this cap, unplayable lag or "DPS Lag" begins. Later on, a Dev called Phax admits Vanshilars thread discovered the issue for them. He then says the bandwidth cap will be going up. Theres also an interesting tidbit on why Dungeon Alert exists.


Two weeks after Vanshilars thread, the Dev Phax requests players to test out the new bandwidth cap and other changes on Lamannia.


This lead up to the release of Update 5 (June.28.2010) that states two important things:


Quote:
Improvements have been made to server and network that should greatly enhance performance in heavy combat situations. Happy shrouding everyone!

Quote:
The following changes have been made to combat for better game performance and balance:
  • Attacks with two handed weapons performed while moving no longer perform Glancing Blows.
       
  • Two Weapon Fighting attacks (weapons or unarmed) have been optimized for performance and balance. Off hand attacks now have a chance to proc (trigger) off of any main hand attacks based on the number of two weapon fighting feats (and related enhancements) you possess. (Instead of being predetermined on certain attacks in the attack chain.)
          
  • A character with no two weapon fighting feats has a base 20% chance to proc off-hand attacks.
          
  • The Two Weapon Fighting, Improved Two Weapon Fighting, and Greater Two Weapon Fighting feats now grant a +20% bonus to proc off-hand attacks.
           
  • Ranger Tempest I and II now grant a +10%/+20% bonus to the chance to proc off-hand attacks when dual wielding instead of a bonus to attack speed.
  • Monk Air Stance and Ranger Tempest I and II now stack with each other for these purposes.

For context on the TWF nerf: Players who had the GTWF Feat before this update had a 100% chance on off-hand procs. Rangers who had the Tempest Prestige 3 Enhancement got an additional off-hand proc per swing. Making their off-hand proc 2 times per swing. TLDR; TWF builds lost 20% off-hand proc chance. TWF Rangers and Fighters got cucked even more. Click here to see the 189 page thread discussing this nerf.   


A day after Update 5s release, players Cyr and Symera try to red pill players who mistakenly think the TWF nerf fixed the lag.


TLDR: "DPS Lag" was a massive issue. Raids were unplayable and the players were having enough of it. A stigma was placed on TWF from players and Devs due to the extra attacks/procs. TWF Players joined together in hopes of finding the cause of the lag (some devs joined in to monitor). Devs mistakenly blame TWF for being the cause so they plan on reducing off-hand proc chance by 20% in the next update. A player, Vanshilar, points out the cause of the "DPS Lag" was due to item procs and ultimately, the server bandwidth cap. Devs increase server bandwidth cap on Lamannia and request players test it. "DPS Lag" greatly reduced due to increased server bandwidth cap. Devs continue planned TWF nerf under the guise of fixing "lag".


CONCLUSION: Turbine was too cheap to pay for better servers. Especially in regards to Symera's comment about EU servers having a bandwidth cap of roughly 27800 bps (compared to NAs 20500) and being mostly unaffected by "DPS Lag". I can also confirm Update 5's slight increase in Bandwidth Cap didnt solve the issue outright. I remember countless amounts of unplayable lag wipes in raids during 2011 and 2012. Furthermore, the Devs would rather nerf or outright remove game systems/features in hopes of reducing lag than directly upgrading the server itself.


« Last Edit: Jan 11th, 2020 at 3:38pm by Rose-tinted Goggles »  

Update 1-13: The golden age of DDO

rest wrote on Oct 26th, 2012 at 12:53pm:
DDO is still the old fat wife I have at home who I can't fucking stand, but we've been together so long I can't see myself leaving.
Back to top
 
IP Logged
 
Carpone
Puppy Farmer
****
Offline


I Love Drama!

Posts: 1695
Joined: Jul 2nd, 2011
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #1 - Jan 9th, 2020 at 8:52am
Print Post  
This game needs more players like Vanshilar. That dude figured out many nuanced details about the game like Harry's HP, and some things which weren't widely shared like blind running Abbott tiles, jumping over invis walls in Shroud p1 to ransack the chests for guild renown, our pre-ED 7min ToD completion, and how to afk farm epic scrolls when those things mattered. He was one of the most humble and skilled players, and it was a huge loss when he quit 7 years ago.
  
Back to top
 
IP Logged
 
Artorias
Shroud Slacker
***
Offline



Posts: 1213
Joined: Mar 21st, 2015
Re: DDO and Lag: A History Lesson
Reply #2 - Jan 9th, 2020 at 9:29am
Print Post  
@Rose
Nice post but iirc it is hardly the only in game system to have suffered and nerfed as an excuse for balance and/or game health. TWF got the shaft because they don't want to invest in better servers, they don't think it's financially good to get better equipment  to solve issues that have been plaguing the game for years and it speaks volumes for the confidence they put in their own work. I can play warframe while keeping netflix on background and listening music video on youtube and not lag in the slightest (only tried solo but it's still noteworthy). You could get a private server and play offline and somehow you would still freaking lag

@Carpone
Sounds like quite the guy. Wish I had to chance to meet/play with him. What server was he on?
  
Back to top
 
IP Logged
 
Carpone
Puppy Farmer
****
Offline


I Love Drama!

Posts: 1695
Joined: Jul 2nd, 2011
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #3 - Jan 9th, 2020 at 9:58am
Print Post  
Orien. He and his brother were in Over Raided.
  
Back to top
 
IP Logged
 
Technomage
Epic Poster
*****
Offline


And so it begins...

Posts: 4444
Location: Tucson, AZ
Joined: Aug 27th, 2015
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #4 - Jan 9th, 2020 at 12:56pm
Print Post  
Carpone wrote on Jan 9th, 2020 at 9:58am:
Orien. He and his brother were in Over Raided.

Was his brother Oddlived by any chance?

That guy was REALLY good.
« Last Edit: Jan 9th, 2020 at 12:57pm by Technomage »  

Do not try the patience of wizards, for they are subtle and quick to anger.

We are dreamers, shapers, singers, and makers. We study the mysteries of laser and circuit, crystal and scanner, holographic demons and invocation of equations. These are the tools we employ, and we know many things.

The absence of evidence is not evidence of absence.
Back to top
 
IP Logged
 
Carpone
Puppy Farmer
****
Offline


I Love Drama!

Posts: 1695
Joined: Jul 2nd, 2011
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #5 - Jan 9th, 2020 at 1:27pm
Print Post  
Technomage wrote on Jan 9th, 2020 at 12:56pm:
Was his brother Oddlived by any chance?

That guy was REALLY good.

Vanshilar played Ua (barb) and Bsod (monk).  His brother played Glum (bard), and a bunch of other alts with four letter names.  They're still in Over Raided, but the guild has been dead since 2014 when many stopped playing due to boredom (Shataan), xfered to Omnipresence (Nixvali), or joined an active Orien guild.

OR was a small guild with about 15 active accounts at any given time.  There's only one active guy in OR (Flamicia) who came back a month ago after a six year hiatus.  Titanticus (founder, original guild leader, and first completionist on Orien) came back about eight months ago for a month when some RL friends were interested in playing.  They didn't stick around.  There's five of us from old school OR days that still play.

There, that's more info than you ever wanted to know about Over Raided.
« Last Edit: Jan 9th, 2020 at 1:28pm by Carpone »  
Back to top
 
IP Logged
 
Asheras
Completionist (i.t.p.)
******
Offline


This is why we can't have
nice things.

Posts: 10230
Location: Ohio
Joined: Jun 9th, 2010
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #6 - Jan 9th, 2020 at 5:13pm
Print Post  
Those were good times.   That TWF nerf brought the tears and drama.
  
Back to top
 
IP Logged
 
Rose-tinted Goggles
Waterworks Kobold
**
Offline


I Love Nostalgia!

Posts: 191
Joined: Jun 3rd, 2018
Re: DDO and Lag: A History Lesson
Reply #7 - Jan 9th, 2020 at 9:02pm
Print Post  
Carpone wrote on Jan 9th, 2020 at 8:52am:
This game needs more players like Vanshilar. That dude figured out many nuanced details about the game like Harry's HP, and some things which weren't widely shared like blind running Abbott tiles, jumping over invis walls in Shroud p1 to ransack the chests for guild renown, our pre-ED 7min ToD completion, and how to afk farm epic scrolls when those things mattered. He was one of the most humble and skilled players, and it was a huge loss when he quit 7 years ago.


His Attack Speed Formulas helped greatly in DPS calculators too. A true legend. I also enjoyed reading Sirgogs forum posts back then if anyone remembers him. He also quit shortly after MotU. 


Artorias wrote on Jan 9th, 2020 at 9:29am:
@Rose
Nice post but iirc it is hardly the only in game system to have suffered and nerfed as an excuse for balance and/or game health. TWF got the shaft because they don't want to invest in better servers, they don't think it's financially good to get better equipment  to solve issues that have been plaguing the game for years and it speaks volumes for the confidence they put in their own work.


This. Many game systems got axed for sure. The Fvs Archon ambient light probably being one of them. As they probably disliked players cheesing Rainbow in the Dark.

You're exactly right on them refusing to pay/upgrade servers. Thats the only conclusion I can come to after the recent digging I've done. Especially in regards to Symera's comment about EU servers having a bandwidth cap of roughly 27800 bps (compared to NAs 20500) and being mostly unaffected by "DPS Lag". I can also confirm Update 5's slight increase in Bandwidth Caps didnt solve the issue outright. I remember countless amounts of unplayable lag wipes in raids during 2011 and 2012. I wasnt as cautious as guilds like Over Raided that would test each instance of the raid before committing to a full run as I only ran pugs. Still, we shouldn't have to do that as players.


Asheras wrote on Jan 9th, 2020 at 5:13pm:
Those were good times.   That TWF nerf brought the tears and drama.


Its honestly pretty shady they were hiding behind "lag" to quell most of the outrage. Although at the time of the 189 page outcry thread Eladrin was oblivious to the Bandwidth Cap issues. So maybe he was being genuine. Although they knew about the Bandwidth Cap issue 2-3 weeks later. They decided to release the nerf anyways.

It definitely could have been handled better. Perhaps dramatically increasing bandwidth caps while keeping TWF the same. Then giving double strike and increasing glancing blow damage to THF feats in order to fix the balance issues would have played out better. This assumes Turbine cared about the players, which they clearly didnt.  





 
« Last Edit: Jan 9th, 2020 at 9:33pm by Rose-tinted Goggles »  

Update 1-13: The golden age of DDO

rest wrote on Oct 26th, 2012 at 12:53pm:
DDO is still the old fat wife I have at home who I can't fucking stand, but we've been together so long I can't see myself leaving.
Back to top
 
IP Logged
 
Starkjade
Completionist (i.t.p.)
******
Offline


Rogue Scholar

Posts: 7082
Location: Canadia
Joined: Aug 27th, 2013
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #8 - Jan 10th, 2020 at 7:26am
Print Post  
Rose-tinted Goggles wrote on Jan 9th, 2020 at 9:02pm:
I also enjoyed reading Sirgogs forum posts back then if anyone remembers him.


He does Path of Exile videos now.

Had a bit of a flashback to my golden age of DDO when I came across one in my youtube feed.
  

Two sips from the cup of human kindness and I'm shitfaced
Back to top
 
IP Logged
 
Technomage
Epic Poster
*****
Offline


And so it begins...

Posts: 4444
Location: Tucson, AZ
Joined: Aug 27th, 2015
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #9 - Jan 10th, 2020 at 10:39am
Print Post  
Starkjade wrote on Jan 10th, 2020 at 7:26am:
He does Path of Exile videos now.

Which channel is his? There are a lot.
  

Do not try the patience of wizards, for they are subtle and quick to anger.

We are dreamers, shapers, singers, and makers. We study the mysteries of laser and circuit, crystal and scanner, holographic demons and invocation of equations. These are the tools we employ, and we know many things.

The absence of evidence is not evidence of absence.
Back to top
 
IP Logged
 
Asheras
Completionist (i.t.p.)
******
Offline


This is why we can't have
nice things.

Posts: 10230
Location: Ohio
Joined: Jun 9th, 2010
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #10 - Jan 10th, 2020 at 10:43am
Print Post  
Rose-tinted Goggles wrote on Jan 9th, 2020 at 9:02pm:
Its honestly pretty shady they were hiding behind "lag" to quell most of the outrage. Although at the time of the 189 page outcry thread Eladrin was oblivious to the Bandwidth Cap issues. So maybe he was being genuine. Although they knew about the Bandwidth Cap issue 2-3 weeks later. They decided to release the nerf anyways.

It definitely could have been handled better. Perhaps dramatically increasing bandwidth caps while keeping TWF the same. Then giving double strike and increasing glancing blow damage to THF feats in order to fix the balance issues would have played out better. This assumes Turbine cared about the players, which they clearly didnt.  
 


I think it just shows how pointless all the outrage is in the moment.  At the end of the day, the TWF nerfs had very little impact on the game experience.  U5-U11 was still a high water mark for DDO in a lot of players opinions.  Even those playing TWF builds.  How many points have there really been in the game where a TWF melee build of some type has not been in the top tier for sustained single target DPS?
  
Back to top
 
IP Logged
 
Starkjade
Completionist (i.t.p.)
******
Offline


Rogue Scholar

Posts: 7082
Location: Canadia
Joined: Aug 27th, 2013
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #11 - Jan 10th, 2020 at 5:10pm
Print Post  
Technomage wrote on Jan 10th, 2020 at 10:39am:
Which channel is his? There are a lot.


http://www.youtube.com/channel/UC1SoHjGhDqK6h980sYsXjhg

I mean, I'm really just assuming it's him, as I didn't know him other than through forum posts, but he's Aussie, and has a similar in-depthness to his PoE vids as his forum posts had.
  

Two sips from the cup of human kindness and I'm shitfaced
Back to top
 
IP Logged
 
Meursault
Horoluth Raider
****
Offline


I Love Freedom of Speech

Posts: 2410
Location: Hartford, CT; USA
Joined: Aug 22nd, 2013
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #12 - Jan 10th, 2020 at 5:24pm
Print Post  
Carpone wrote on Jan 9th, 2020 at 8:52am:
This game needs more players like Vanshilar. That dude figured out many nuanced details about the game like Harry's HP, and some things which weren't widely shared like blind running Abbott tiles, jumping over invis walls in Shroud p1 to ransack the chests for guild renown, our pre-ED 7min ToD completion, and how to afk farm epic scrolls when those things mattered. He was one of the most humble and skilled players, and it was a huge loss when he quit 7 years ago.

And devs willing to work with them. Even if they didn't abandon the nerf when they learned it wasn't the real culprit, at least they worked with the community on discovering the problem and were open enough with players to discuss things like server bandwidth cap. Devs now are far worse, too insecure to to admit players might have a good idea they didn't, so they don't listen and don't share and the game suffers for it.

Devs and players could be a team working together to improve the game for the benefit of all concerned.

Hey @Rose, is this going to be a series on the history of lag? This anecdote was fascinating, I'd love to hear more.
  

Turdbin, keep changing the DDO rules, because McDonalds sold over 200 billion hamburgers by changing the recipe for their Special Sauce every couple of months to keep interest up.
Back to top
 
IP Logged
 
Rose-tinted Goggles
Waterworks Kobold
**
Offline


I Love Nostalgia!

Posts: 191
Joined: Jun 3rd, 2018
Re: DDO and Lag: A History Lesson
Reply #13 - Jan 10th, 2020 at 10:16pm
Print Post  
Starkjade wrote on Jan 10th, 2020 at 7:26am:
He does Path of Exile videos now.

Had a bit of a flashback to my golden age of DDO when I came across one in my youtube feed.


Sirgog made a post yesterday, almost two years after his last post. He says if SSG merges the servers he would come back immediately.


Asheras wrote on Jan 10th, 2020 at 10:43am:
I think it just shows how pointless all the outrage is in the moment.  At the end of the day, the TWF nerfs had very little impact on the game experience.  U5-U11 was still a high water mark for DDO in a lot of players opinions.  Even those playing TWF builds.  How many points have there really been in the game where a TWF melee build of some type has not been in the top tier for sustained single target DPS? 


An overreaction for sure. In fact, Vanshilar made a post after Update 8 detailing how the current endgame builds stack up against each other. At 50% Fortification most TWF builds were either tied, or even slightly ahead of THF builds in DPS. Rogue splits were obviously way ahead but they're about to get cucked come Update 11.


Meursault wrote on Jan 10th, 2020 at 5:24pm:
Hey @Rose, is this going to be a series on the history of lag? This anecdote was fascinating, I'd love to hear more.


I currently have nothing planned. In fact, it was an accident this topic happened. I stumbled upon a comment on a thread then entered the rabbit hole from there. Further installments of this topic would be difficult as I think the Devs have become less forthcoming as time went on. It probably wouldnt be as interesting either. The sad truth is, the majority of the lag issues plaguing DDO are mostly due to Turbine/SSG being too cheap to pay for proper servers. A secondary to that probably being shit code/game engine.

I think it would be interesting finding all the game systems the Devs nerfed or axed throughout the years because of "lag". Like the removal of the Fvs Archon ambient light. Which was an actual lore thing and was a great addition to the game. 

It took some digging but I found a comment on why it was removed. I cant find the exact date it was removed but I thought sometime around late 2011 or 2012.

Heres the quote:

Quote:
For you (and perhaps many!) that may be true, though without the right tools gauging whether or not a change on our end mitigated lag on your end is subjective at best.

To be clear: The Lantern Archons' lights fundamentally broke DDO's lighting system in almost all content, especially if multiple players had them out at once. They should not have ever had a light. When we determined that this also caused severe performance problems on lower-end hardware, the decision was made to cut the lights (which, per how the engine works, should not have been there in the first place.) It did not cause lag for everybody, but it DID cause severe performance issues for some, and caused other kinds of problems game-wide.

There are lots of kinds of "lag". Some affect everyone, some affect only certain kinds of hardware. Sometimes it's server-to-client issues, sometimes it's client-side but game-related. Ultimately, when we make a change to improve performance, it is because itis, concretely, affecting a portion of the playerbase. Even if that is not every player.
 

TLDR: Archons allowed players to cheese Rainbow in the Dark and also caused some players performance issues because they play on potatoes. So we removed it.

« Last Edit: Jan 10th, 2020 at 10:58pm by Rose-tinted Goggles »  

Update 1-13: The golden age of DDO

rest wrote on Oct 26th, 2012 at 12:53pm:
DDO is still the old fat wife I have at home who I can't fucking stand, but we've been together so long I can't see myself leaving.
Back to top
 
IP Logged
 
Flav
Vault Frog
*
Offline


One Frog to Rule them
All!

Posts: 9985
Location: Land of the Frogs
Joined: Aug 29th, 2010
Gender: Male
Re: DDO and Lag: A History Lesson
Reply #14 - Jan 12th, 2020 at 6:29am
Print Post  
Dungeon Alert and Dungeon Scaling have resolved all lag issues.... And that occured long ago.
There's no lag in DDO.

Roll Eyes
  

Yes my avatar is an Hermine eating a Greenland Lemming for brunch.
Back to top
 
IP Logged
 
Dark_Helmet
Wielder of the Schwartz
****
Offline


I hate you!

Posts: 1176
Joined: Feb 14th, 2010
Re: DDO and Lag: A History Lesson
Reply #15 - Jan 13th, 2020 at 2:33am
Print Post  
I really wish the older devs came back and post about the internal squabbles... They introduced a couple of new lag types going back to 2008 (shroud).  TWF was just a symptom of an issue that was able to be defended with math.

It was amazing how their management wouldn't even acknowledge that it was possible to cause lag just by running up to a chest (the shroud population of chest loot).  The point haired boss insulted his players numerous times... amazing how he was finally gone in 2012... and a certain lag got reduced?  Conspiracy theory 101!  Just like lag and server pops, they MUST be related.  Wink



  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1
Send TopicPrint