Thread: Very big .res files

Page 2 of 2 FirstFirst 12
Results 26 to 38 of 38
  1. #26
    Still learning wolf-3d's Avatar  
    Tester
    Join Date
    Jan 2011
    Location
    Somewhere Warm.
    Posts
    1,431

    Re: Very big .res files

    Quote Originally Posted by Solokiller View Post
    I did. I made my case for focusing on stability, performance and error resistance, as well as dealing with issues like large res files, late precaching, and fixing bugs, whether they're major or minor, but they didn't want to do it. No amount of effort can convince those who do not wish to be convinced.
    Yep since Solo's exit, we are back on "Valve Time" and things will get done, when they are done. (unfortunately)
    Regards
    Wolf-3D

  2. #27
    Banned Solokiller's Avatar
    Join Date
    Sep 2004
    Location
    Belgium
    Posts
    3,314

    Re: Very big .res files

    I was able to build a prototype for general purpose network string tables in just a few hours. It's based on Source's system: https://developer.valvesoftware.com/...#String_Tables

    I can network the table from the server to the client, sending only changes. That includes arbitrary binary data.

    This sort of thing could be used to replace the engine's resource lists, enable late precaching (since these tables can be added to at any time), replace the map cycle (limited because it's sent as a series of network messages), and eliminate the need for a sound cache.

    Obviously, putting together a fully functional solution isn't that easy, but as a prototype, it works. I didn't figure out how to send large amounts of data yet, so it can still cause reliable channel overflows, but i'm sure there's a way to avoid that. I'm thinking something like this:

    Code:
    void CServer_Client::SendServerData()
    {
    	byte byteBuffer[ 100000 ];
    
    	//A network buffer that has 100000 bytes worth of space
    	CNetworkBuffer buffer( "serverDataBuf", byteBuffer, sizeof( byteBuffer ) );
    
    	//start parameter is a reference here so the table can update it
    	if( currentTable->Write( buffer, start ) )
    	{
    		//The table has finished writing all of its contents to the buffer
    		currentTable = GetNextTable();
    		start = 0;
    	}
    
    	if( currentTable == nullptr )
    	{
    		m_ConnectionState = STATE::TABLESSENT;
    	}
    
    	//Send the data
    	m_NetworkChannel->Send( buffer );
    }
    You'd send each table one at a time, sending as much as possible. This way, reliable channel overflows should never happen, unless you allow strings and/or binary data to become larger than the buffer size (insert "are you mad" here).
    Because this happens on client connect, you don't have to worry about packet fragmentation or sending too much data, since they're not in-game and won't notice it. As long as they receive the data without too many packet drops, it should be fine.

  3. #28
    Banned
    Join Date
    Mar 2016
    Posts
    21

    Re: Very big .res files

    Thanks Solokiller! I Pm'd Sniper and asked him to fix the issue. Please fix it Sniper

  4. #29
    Banned Solokiller's Avatar
    Join Date
    Sep 2004
    Location
    Belgium
    Posts
    3,314

    Re: Very big .res files

    I doubt that will happen. I first suggested we use network string tables around a year ago, and brought it up on several occasions. He didn't want to use it. I don't know what - if anything - he's planning to do about this, but he won't tell anybody, that's for sure.

  5. #30
    Administrator JPolito's Avatar  
    Manager
    Join Date
    Apr 2004
    Posts
    7,509

    Re: Very big .res files

    Quote Originally Posted by Solokiller View Post
    When last i raised the issue last week, it didn't seem that important to the team. I believe the response was something along the lines of cool features being more important.
    No one actually said that.

  6. #31
    Administrator Sniper's Avatar  
    Manager
    Join Date
    Aug 2001
    Posts
    7,102

    Re: Very big .res files

    Solokiller is no longer a team member. He doesn't speak for the team.
    We'll take a look at this and fix the issue.

    Thanks for reporting it.
    - David "Sniper" McDermott
    Sven Co-op Lead Programmer
    "I mean it's like a koala bear crapped a rainbow in my brain!"

  7. #32
    Mega Kiwi Untamed's Avatar
    Join Date
    Jun 2009
    Location
    New Zealand
    Posts
    17

    Re: Very big .res files

    Appreciated
    Thanks guys

  8. #33
    Administrator Sniper's Avatar  
    Manager
    Join Date
    Aug 2001
    Posts
    7,102

    Re: Very big .res files

    Quote Originally Posted by Sniper View Post
    Solokiller is no longer a team member. He doesn't speak for the team.
    We'll take a look at this and fix the issue.

    Thanks for reporting it.

    This actually took a fair bit of effort to debug / troubleshoot.

    I commented earlier today on Discord:

    1.) Res files can have longer filenames for (e.g. a file is located under a long directory path)
    2.) Spaces and parentheses are now properly accounted for in filenames, without breaking the download system
    3.) Res files can be much longer thanks to a larger buffer (this isn't 100% tested yet)

    This update should be available in the next patch after some testing is completed.

    Yes, we focused on other issues since it was first reported, but hopefully this should now be resolved.
    - David "Sniper" McDermott
    Sven Co-op Lead Programmer
    "I mean it's like a koala bear crapped a rainbow in my brain!"

  9. #34
    func_vehicle enthusiaist w00tguy123's Avatar
    Join Date
    Dec 2006
    Location
    U.S. West
    Posts
    1,544

    Re: Very big .res files

    This is still a problem. I have about 1k files precached currently and am getting the "SZ_GetSpace: overflow without FSB_ALLOWOVERFLOW" crash.

    Shortening file paths helps, but they would all have to be 8 characters or less if I wanted to use up the 8k precache slots (impossible even if I use paths like "A/BBB.spr").
    Love,
    w00tguy

  10. #35
    Registered User
    Join Date
    Mar 2005
    Posts
    189

    Re: Very big .res files

    Dam, I guess this whole "SC team being incompetent" thing goes back for years behind the scenes.

    Doing like some Temp[512] to Temp[1028] and a few parse changes are not rly a 'fix' compared with you possibly updating the whole system to allow late precaching and other benefits owel.. just my 3 cents. I guess if I was involved I'd prolly have been banned too even if I had hypothetically made a whole solution.

    reminds me of good ol' leftist politics and why they've been failing as of late.
    Last edited by HL2 RULEZ; 09-01-2018 at 09:21 PM.
    BLEH. BLEHEG
    BLEHHEGLAH!!!!!!!!! O.o

  11. #36
    Administrator AdamR's Avatar  
    Manager
    Join Date
    Mar 2004
    Location
    Cardiff, South Wales [uk]
    Posts
    8,402

    Re: Very big .res files

    Quote Originally Posted by HL2 RULEZ View Post
    Dam, I guess this whole "SC team being incompetent" thing goes back for years behind the scenes.

    Doing like some Temp[512] to Temp[1028] and a few parse changes are not rly a 'fix' compared with you possibly updating the whole system to allow late precaching and other benefits owel.. just my 3 cents. I guess if I was involved I'd prolly have been banned too even if I had hypothetically made a whole solution.

    reminds me of good ol' leftist politics and why they've been failing as of late.
    Well, why don't you get involved instead of the common attitude of demanding a gold standard then crapping all over the only people doing any of the work? Come and show us what you can do.
    Last edited by AdamR; 10-01-2018 at 05:14 AM.
    Adam "Adambean" Reece
    Sven Co-op team

    Also on: Steam | Facebook | Twitter | YouTube | Twitch
    Released AMXX plug-ins: Bind number slots | NextMap with Sven Co-op fix | Sven Co-op administrator icons

  12. #37
    Registered User
    Join Date
    Mar 2005
    Posts
    189

    Re: Very big .res files

    'Cuz I'd be banned apparently and in the databse it'd probably say "ARROGANT DIDN'T WORK WITH TEAM" when in reality it'd be like "was too smart, made a system in 1 day and then got banned".

    Not sure how the meaning escaped you when it's in pretty clearly written English.
    BLEH. BLEHEG
    BLEHHEGLAH!!!!!!!!! O.o

  13. #38
    Administrator AdamR's Avatar  
    Manager
    Join Date
    Mar 2004
    Location
    Cardiff, South Wales [uk]
    Posts
    8,402

    Re: Very big .res files

    Oh well, I offered to take a contribution from you.
    Adam "Adambean" Reece
    Sven Co-op team

    Also on: Steam | Facebook | Twitter | YouTube | Twitch
    Released AMXX plug-ins: Bind number slots | NextMap with Sven Co-op fix | Sven Co-op administrator icons

Page 2 of 2 FirstFirst 12

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •