We can avoid multiple ticket additions for the same ChunkTasks
instance. This will help in situations where significant number of
block updates occur for the same chunk in the same tick, such as
water draining.
If an entity is removed while updating an entity slice, then the
iteration over the entity slice's entities could throw a cryptic
exception. Instead, it is better to prevent the entity removal
with a useful log message.
Fixes https://github.com/PaperMC/Paper/issues/9464
Since the packet is broadcasted for players, it means that
the packet will be sent to multiple players. In this case,
modifying the data results in a possible race condition
where a CME may occur as the packet will be serialized on
many different netty IO threads.
It shouldn't cost too much more to check the correct shape,
provided that it is cached and we use the overall AABB
to collect possible entities to check against.
The issues with the old check code is that it will use two
getEntitiesOfClass calls plus the addition of streams
_and_ the toAabbs() logic on VoxelShape. The new code
caches toAabbs, uses one getEntitiesOfClass call, and
does not use streams. Then compared to 1.12, we are
only performing two additional AABB checks per item.
Forgot to update the distandce field in the single user area map
on update, and additionally used the wrong radius (introduced
last commit) for the broadcast map
Upstream has released updates that appear to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing
CraftBukkit Changes:
fd92f1e65 SPIGOT-7378: Add BlockDropItemEvent for Suspicious Sand & Gravel drops
cb1b69d13 SPIGOT-7377: Server sends player list twice
Spigot Changes:
16cfc987 Rebuild patches
For whatever reason, vanilla does not mark the chunk as
dirty when changing its tick lists.
We also have it return dirty if the time since the last
save has changed, since it would affect the tick offsets
in the ticklist.
CB used the dirty flag to construct the chunk unload event,
but then sets mustNotSave to the inverted value of the event
after calling the event without considering that the chunk may
actually be brought up to loaded status again later. Then, CB
overrides the isUnsaved method of LevelChunk to additionally
use mustNotSave.
Thus, if the chunk is not marked dirty when unloading, the
mustNotSave value will be set to true. Then, once the chunk
is reloaded and edited the dirty flag will be set. However,
when unloading the chunk finally, the isUnsaved method
will return false due to mustNotSave being true. Thus, the
chunk will never be saved.
To fix these issues, no longer make mustNotSave override
isUnsaved and always set the save flag for the chunk unload
event.
This issue started popping up recently due to the recent
change to mark chunks as not dirty after saving them, which
increased the chance of the save issue occurring in the first
place.
When we update the chunk state to border, it should be
the case that isChunkLoaded returns true and that
getChunkIfLoadedImmediately returns a non-null value.
Now add the chunk to the loaded map before making any
callbacks after updating to border state.
Patch documentation to come
Issues with the old system that are fixed now:
- World generation does not scale with cpu cores effectively.
- Relies on the main thread for scheduling and maintaining chunk state, dropping chunk load/generate rates at lower tps.
- Unreliable prioritisation of chunk gen/load calls that block the main thread.
- Shutdown logic is utterly unreliable, as it has to wait for all chunks to unload - is it guaranteed that the chunk system is in a state on shutdown that it can reliably do this? Watchdog shutdown also typically failed due to thread checks, which is now resolved.
- Saving of data is not unified (i.e can save chunk data without saving entity data, poses problems for desync if shutdown is really abnormal.
- Entities are not loaded with chunks. This caused quite a bit of headache for Chunk#getEntities API, but now the new chunk system loads entities with chunks so that they are ready whenever the chunk loads in. Effectively brings the behavior back to 1.16 era, but still storing entities in their own separate regionfiles.
The above list is not complete. The patch documentation will complete it.
New chunk system hard relies on starlight and dataconverter, and most importantly the new concurrent utilities in ConcurrentUtil.
Some of the old async chunk i/o interface (i.e the old file io thread reroutes _some_ calls to the new file io thread) is kept for plugin compat reasons. It will be removed in the next major version of minecraft.
The old legacy chunk system patches have been moved to the removed folder in case we need them again.
Logging into an unloaded world isn't going to end well.
This may fix the cases of people seeing errors about regionfiles
being closed, as loading chunks in an unloaded world will cause this
as the regionfile cache is closed but not cleared.
These can be called while an entity is being added to the world,
and if the entity is being added from a chunk load context the
sync load will block indefinitely (because the chunk load context
is for completing the chunk to FULL).
This does raise questions about the current system for these
dynamic registrations, as it looks like there is _zero_ logic
to account for the case where the chunk is _not_ currently loaded
and then later loaded.
Configurable under "settings.chunk-loading.player-max-chunk-load-rate",
defaults to -1. This commit also changes the chunk loading to be
distributed equally for all players, rather than distance based. This is
to ensure players flying around do not take priority over everyone else.
The exception to this new rule is the min-load-radius, which still has
priority over everything else.
The worldgen thread will attempt to get structure references
via the world's getChunkAt method, which is fine if the gen is
not cancelled - but if the chunk was unloaded, the call will block
indefinitely. Instead of using the world state, we use the already
supplied generatoraccess which will always have the chunk available.
Missed patch from tuinity merge
- Save level.dat first, in case the shutdown is killed later
- Force run minecraftserver tasks and the chunk source tasks
while waiting for the chunk system to empty, as there's simply
too much trash that could prevent them from executing during
the chunk source tick (i.e "time left in tick" logic).
Needs one for ItemStack and Entity. It looks like I missed
the entity one because Mojang defined it as being simple,
which it certainly isn't (they "missed" it too).
- Port player chunk loader patch
Makes the chunk system act as it did in 1.17, no additional tickets (and thus logic) to make a chunk ticking.
Adds simulation distance API, deprecates old no-tick method.
- More collision optimisations
Ancient patch from tuinity that never could be pushed to master.
- Fix Optimise ArraySetSorted#removeIf patch
- Execute chunk tasks fairly for worlds while waiting for next tick
- Port Replace ticket level propagator
The data provided is always from the regionfile thread, which
does not copy the data out. So if two separate calls need
the data, then there's going to be a problem.
These used to be here from Spigot, but were dropped with 1.17.
Now in 1.17, this state is _even more_ critical than it was before,
so these must exist to catch stupid plugins.
If the chunk count is high, then the memory used by the
chunks adds up and could cause problems. By flushing
every so many chunks, the server will not become
stressed for memory. It will also not increase the total
time to save, as flush saving performs a full flush at
the end anyways.
The logic cannot even determine what local chunk they should be,
and out of bounds chunks can only occur from external modifications
to the regionfile.
If regionfile recalculation cannot occur, then do not attempt
to retry read actions.
Fixes https://github.com/PaperMC/Paper/issues/6718
Note from Spottedleaf:
Please note that this doesn't fix any problems using dataconverter
because dataconverter does not re-route world gen settings conversion.
Co-authored-by: Callum Seabrook <callum.seabrook@prevarinite.com>
This PR contains all of Tuinity's patches. Very notable ones are:
- Highly optimised collisions
- Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16)
- Starlight https://github.com/PaperMC/Starlight
- Rewritten dataconverter system https://github.com/PaperMC/DataConverter
- Random block ticking optimisation (wrongly dropped from Paper 1.17)
- Chunk ticking optimisations
- Anything else I've forgotten in the 60 or so patches
If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml.
* Port tuinity, initial patchset
* Update gradle to 7.2
jmp said it fixes rebuildpatches not working for me. it fucking better
* Completely clean apply
* Remove tuinity config, add per player api patch
* Remove paper reobf mappings patch
* Properly update gradlew
* Force clean rebuild
* Mark fixups
Comments and ATs still need to be done
* grep -r "Tuinity"
* Fixup
* Ensure gameprofile lastaccess is written only under the state lock
* update URL for dataconverter
* Only clean rebuild tuinity patches
might fix merge conflicts
* Use UTF-8 for gradlew
* Clean rb patches again
* Convert block ids used as item ids
Neither the converters of pre 1.13 nor DFU handled these cases,
as by the time they were written the game at the time didn't
consider these ids valid - they would be air. Because of this,
some worlds have logspam since only DataConverter (not DFU or
legacy converters) will warn when an invalid id has been
seen.
While quite a few do need to now be considered as air, quite a lot
do not. So it makes sense to add conversion for these items, instead
of simply suppressing or ignoring the logs. I've now added id -> string conversion
for all block ids that could be used as items that existed in the game
before 1.7.10 (I have no interest in tracking down the
exact version block ids stopped working) that were on
https://minecraft-ids.grahamedgecombe.com/
Items that did not directly convert to new items will
be instead converted to air: stems, wheat crops, piston head,
tripwire wire block
* Fix LightPopulated parsing in V1466
The DFU code was checking if the number existed, not if it
didn't exist. I misread the original code.
* Always parse protochunk light sources unless it is marked as non-lit
Chunks not marked as lit will always go through the light engine,
so they should always have their block sources parsed.
* Update custom names to JSON for players
Missed this fix from CB, as it was inside
the DataFixers class.
I decided to double check all of the CB changes again:
DataFixers.java was the only area I missed, as I had inspected all
datafixer diffs and implemented them all into DataConverter. I also
checked Bootstrap.java again, and re-evaluated their changes. I had
previously done this, but determined that they were all bad.
The changes to make standing_sign block map to oak_sign block in
V1450 is bad, because that's not the item id V1450 accepts. Only
in 1.14 did oak_sign even exist, and as expected there is a converter
to rename all existing sign items/blocks.
The fix to register the portal block under id 1440 is useless, as
the flattenning logic will default to the lowest registered id - which
is the exact blockstate that CB registers into 1440. So it just
doesn't do anything.
The extra item ids in the id -> string converter are already added,
but I found this from EMC originally.
The change for the spawn egg id 23 -> Arrow is just wrong,
that id DOES correspond to TippedArrow, NOT Arrow. As
expected, the spawn egg already has a dedicated mapping for
Arrow, which is id 10 - which was Arrow's entity id.
I also ported a fix for the cooked_fished id update. This doesn't
really matter since there is already a dataconverter to fix this,
but the game didn't accept cooked_fished at the time. So I see
no harm.
* Review all converters and walkers
- Refactor V99 to have helper methods for defining entity/tile
entity types
- Automatically namespace all ids that should be namespaced.
While vanilla never saved non-namespaced data for things that
are namespaced, plugins/users might have.
- Synchronised the identity ensure map in HelperBlockFlatteningV1450
- Code style consistency
- Add missing log warning in V102 for ITEM_NAME type conversion
- Use getBoolean instead of getByte
- Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow
rename, as it will affect ENTITY_NAME type
- Always set isVillager to false in V502 for Zombie
- Register V808's converter under subversion 1 like DFU
- Register a breakpoint for V1.17.1. In the future, all final
versions of major releases will have a breakpoint so that
the work required to determine if a converter needs a breakpoint
is minimal
- Validate that a dataconverter is only registered for a version
that is registered
- ConverterFlattenTileEntity is actually ConverterFlattenEntity
It even registered the converters under TILE_ENTITY, instead of
ENTITY.
- Fix id comparison in V1492 STRUCTURE_FEATURE renamer
- Use ConverterAbstractStatsRename for V1510 stats renamer
At the time I had written that class, the abstract renamer didn't
exist.
- Ensure OwnerUUID is at least set to empty string in
V1904 if the ocelot is converted to a cat (this is
likely so that it retains a collar)
- Use generic read/write for Records in V1946
Records is actually a list, not a map. So reading map was
invalid.
* Always set light to zero when propagating decrease
This fixes an almost infinite loop where light values
would be spam queued on a very small subset on blocks.
This also likely fixes the memory issues people were
seeing.
* re-organize patches
* Apply and fix conflicts
* Revert some patches
getChunkAt retains chunks so that plugins don't spam loads
revert mc-4 fix will remain unless issues pop up
* Shuffle iterated chunks if per player is not enabled
Can help with some mob spawning stacking up at locations
* Make per player default, migrate all configs
* Adjust comments in fixups
* Rework config for player chunk loader
Old config is not compatible. Move all configs to be
under `settings` in paper.yml
The player chunk loader has been modified to
less aggressively load chunks, but to send
chunks at higher rates compared to tuinity. There are
new config entries to tune this behavior.
* Add back old constructor to CompressionEncoder/Decoder (fixes
Tuinity #358)
* Raise chunk loading default limits
* Reduce worldgen thread workers for lower core count cpus
* Raise limits for chunk loading config
Also place it under `chunk-loading`
* Disable max chunk send rate by default
* Fix conflicts and rebuild patches
* Drop default send rate again
Appears to be still causing problems for no known reason
* Raise chunk send limits to 100 per player
While a low limit fixes ping issues for some people, most people
do not suffer from this issue and thus should not suffer from
an extremely slow load-in rate.
* Rebase part 1
Autosquash the fixups
* Move not implemented up
* Fixup mc-dev fixes
Missed this one
* Rebase per player viewdistance api into the original api patch
* Remove old light engine patch part 1
The prioritisation must be kept from it, so that part
has been rebased into the priority patch.
Part 2 will deal with rebasing all of the patches _after_
* Rebase remaining patches for old light patch removal
* Remove other mid tick patch
* Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch
Replaced by `Do not copy visible chunks`
* Revert AT for Vec3i setX/Y/Z
The class is immutable. set should not be exposed
* Remove old IntegerUtil class
* Replace old CraftChunk#getEntities patch
* Remove import for SWMRNibbleArray in ChunkAccess
* Finished merge checklist
* Remove ensureTickThread impl in urgency patch
Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com>
Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
* Ensure priority updates are determined by current level, not deferred level
Deferral could cause an infinite loop to occur.
Also, re-add the light engine priority logic.
* Rebase
Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com>
The chunk system can swallow the exception given it's all
built with completablefuture, so ensure it is at least printed.
Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com>
Mojang did NOT add dataconverters for world gen configurations
that they CHANGED. So, the codec fails to parse old data.
This fixes two instances:
- IntProvider is new and Mojang did not account for old data.
Thankfully, only ColumnPlace needed to be special cased.
- TreeConfiguration had changes. Thankfully, they were
only renames for one value and thankfully defaults could
be provided for two new values (WITHOUT changing behavior).
even even even even even even even even even even
even even even even even even even even even even
even even even even even even even even even even
even even even even even even even even even even
even even even even even even even even even even
even even even even even even even even even even
more work
Spottedleaf