Skoodge wrote on Jul 31
st, 2019 at 3:51pm:
Thank you for clarifying and I want to say up front that I like you. Since that is a sincere sentiment I’m going to say it twice just so you know how much I mean it.
I like you.
You’re wrong and your proposed system isn’t realistic, but I really do like you.
First off and most importantly – I’m fairly certain they don’t have the systems in place to gather the information they would need to make your proposal a reality. Even if they did it would take a great deal of man power weeks and weeks to process and analyze. Even if they had the budget for it, I wouldn’t recommend they put forth that much effort when a much simpler solution is on the table.
Not only is the yield verse gain aspect well out of proportion with your ideas, it still wouldn’t help them with new quests. They’d still have to start with rough ideas for the XP per quest then have to go back in to readjust as new data (which they still can’t track and doesn’t exist) was added at a later date. Your idea takes far more time to implement and promises to take far more time each time a new pack is introduced…if they could even monitor what you want them to…which I’m still pretty sure they can’t.
Secondly, there’s no such thing as an average party. In fact none of your parameters consistently exist to even be measured. Outside of the newest packs, there’s no such thing as a party in DDO anymore. If you get 3 guys and a few hires for most (not all, most) of the content, you’re doing lucky.
So who do we even monitor to get our average? A party of 3 newbs wandering around aimlessly forever? Or a party of 3 long time players because the average quest time is going to be the same no matter what the quest – 5 to 10 minutes max. Either way your sample is flawed and your data is worthless.
By your proposal Wiz King becomes a 5 minute quest – because that’s what your findings are going to show (if they existed, which they don’t). It’s a 5 minute quest and the XP would be nerfed as such. Mirror is a 5 minute quest. WGU is a 10 minute quest. Spies is an 8 minute quest. Do we drop the xp in spies dramatically for someone trying to run it at level because it’s mostly run quicker by toons 8 levels over on a daily basis?
And how do we accumulate the non-existent data for dead quests and packs? How do we know how long it takes to run a quest that no one runs?
No offense but I’ve spent weeks thinking about my system and ironing out all the wrinkles based on facts and taking the companies goals and man power situation into account.
Creating a base XP rate on the size of the dungeon and allowing it to expand by optionals still gives a good rate for zergers and allows the daisy sniffers to profit by taking longer in the quest is far easier to implement than what you’re bringing to the table.
I like you too, and I like that we can have this conversation, even if it's a tree falling in the forest. It doesn't bruise my ego if I am wrong, and I admit it's possible. I don't think anybody ever makes real progress by sticking to an idea just because they are the one that thought of it - it's far better to share ideas, debate their relative merit, and try (if possible) the best one even if it's not yours.
But I'm not quite ready to admit that I'm wrong here.
You do raise one point that might pose a problem for my idea. You say they don’t have the systems in place to gather the information they would need to make my proposal a reality, but Sev said "We have been analyzing our data on quests, completion times, and completion numbers". I admit that it's possible that "his numbers" are 3 runs with Cordo and spying on one group on a Friday night. But it's not unreasonable to think that it means they've been logging basic quest completion statistics.
On a server with fewer than 500 concurrent users and quests taking at a minimum several minutes, the data load for completion stat logging is perfectly reasonable. And we know some logging used to take place because you could see your character quest history in the old web API.
So while it's entirely possible that Sev was full of shit about having data, could we set that aside as "maybe" and evaluate the rest of my proposal? Because if you do this for a living, you might find something that helps you with a customer that *does* collect good data about their game. Maybe I'm naive to think this crew is capable of such competence, but their money is on the line, maybe they've started doing something right.
Your question about the non-existent data for dead quests and packs and new packs is also a reasonable concern, my method does require a fairly large sample to be effective. But we could use your method as an approximation for both and refine it as more data comes in (and if it doesn't for older packs then the error means nothing anyway). Now I know you're thinking about the manpower thing when I say that, but read the next paragraph first.
You say it would take tremendous manpower, but it wouldn't. You do this for a living, but what I do for a living is SQL programming and data analysis. I could write a query that implemented my idea in a few hours (assuming I had the data, but we're letting that slide for now, right?). And once written, that query could be re-run at any time in seconds. Maybe a few minutes, but certainly no longer than that. They could use it to refresh the baseline every 3-6 months to take into account changes to systems or enhancements that changed how easy quests were! It doesn't even have to be done on production servers, the quest activity log could be warehoused nightly in a slow period or at worst on every Tuesday on restart and the analysis done on a development server.
You say they can't collect the data I need, but Sev implied that they already are (see above). All I need is Quest ID, Optionals completed, time in quest, and CharacterID for each party member. Hardly an onerous data collection task. They already display all that and more at the end of the quest if you hit the X key, just log it. I can look up quest details from the database from the QuestID. I can look up character details from the CharacterID. Maybe the character has leveled between questing and warehousing, but one level won't make a difference and multiple levels is unlikely to happen that often and just means a few otherwise valid data points get discarded. With a few month's data, dropping a few completions won't matter.
You say there’s no such thing as an average party, but I'm not looking for an average party, I'm looking for the average *of* parties. I want Average(CompletionTime(Parties-OverLevelParties)) not CompletionTime(Average(Party)), and that's readily available (if you've collected the very basic data). And using a trimmed mean gets rid of the outliers you mention, like the 3 noobs and the 3 long time players you mention.
And by the way, your system is just as susceptible to that - do you classify a quest as short, medium, or long? 3 Noobs give one classification, 3 vets a different. Your method is no better than mine under those circumstances, and enjoys no competitive advantage.
And so what if the party size is 1 or 6? You base it on what people are actually doing, so if the quest is usually run solo then pay it out at the solo average, and if they usually group then pay it at the group rate. For some of each take the average, which is exactly what my method does. If that leads to people suddenly grouping to get a faster rate, what's the problem, it's more grouping.
You ask do we drop the xp in spies dramatically for someone trying to run it at level because it’s mostly run quicker by toons 8 levels over on a daily basis? I assume you're talking about Epic, because in Heroic they would be filtered out as over level, but I say yes, if it's usually run 8 levels higher, then that's how the reward should be calculated. There are plenty of quests people commonly skip and go back to because the rewards don't justify the frustration at level - Proof in the Poison, Stealthy Repossession, etc. It's not worth sweating the 5%, get numbers that work for 95% and you're golden. You're already telling me that when you say your "quick and dirty method" is close enough, we both agree that sweating the details for small differences isn't worth it.
You say that by my proposal Wiz King becomes a 5 minute quest - yes, so set the reward accordingly. It's no longer 10x better than the average quest available? Fine. Sev wants to ransack it into oblivion anyway, and he's not going to take your system and let Wiz King be a relative gold mine. But remember that I agreed with you that optional XP should be paid at quest completion, so under those conditions would WizKing even really be just 5 minutes anymore?
Consider also that changing the rewards might change the dynamic of how it's played, but that's OK too because rerunning my script is basically free so after a few iterations it will reach an equilibrium. Maybe the equation will need a damping factor, but that's no big deal.
You also worry that adding a new pack will invalidate my method or cause an ever increasing amount of work, it won't. I don't need any manual entries for a quest, when it shows up in the log of run quests it gets included in the output with absolutely no modification to my script needed. It's entirely data driven. You may need to add an entry for "pot sweetening" if you want people to favor the new quest over and above the draw of novelty and better loot, but it's not necessary for my system.
And your system isn't even all that different from mine, I'm really just replacing your discrete distribution (6 classifications for quests and 1 for optionals) manually decided on with an continuous distribution automatically calculated by a short script. Increase your number of classifications and your calculations for base XP will approach mine as the number of categories approaches infinity. Treat optionals like Base XP, and they'll approach mine too. Sure it's impractical to do manually, but with a little logging we don't have to do it manually ever again.
With all these reincarnation and reaper benefits, this game is driven by XP per minute and unless you use free market forces like making them all pay the same XP per minute, any attempt to engineer behavior against that is going to both fail and generate resentment.
So yeah, it requires that they do some basic logging, but wouldn't you advise a client to do that anyway? Maybe if Sev has a real fire under his ass today and his logging is a lie he could implement your method today and start collecting data to re-evaluate in the future, but wouldn't having all quests having the same XP per minute be the best way to encourage variety in quests run?
Hey, it's all theory crafting, Sev's not going to listen to either of us, but I've gotten more enjoyment analyzing this with you than the last 5 quests I've run in DDO, so I consider it time well spent. If you're not too frustrated I'd love to hear your thoughts on this, and if you've ever got a client who does appreciate data enough to collect it I'd be happy to chat about how to use it. 10 years ago the tools for big data analysis were rare and expensive, but today they're within reach of even a small cash strapped company.