Previous method only worked for a normal shutdown, and didn't include
when the server enters a closing state due to watchdog crashes
This is the correct variable to detect the server is in the middle of shutdown process
This fix is for the few people who are using such low end systems that
asynchronous chunk loading hurts them rather than helping.
The previous build made paper crash if you turned off async chunks, and
this fixes that issue.
Mark chunks that are blocking main thread for world generation as urgent
Implements a general priority system so that chunks that are sorted in
the generator queues can prioritize certain chunks over another.
Urgent chunks will jump to the front of the line, ensuring that a
sync chunk load on an ungenerated chunk does not lag the server for
a long period of time if the servers generator queues are filled with
lots of chunks already.
This massively reduces the lag spikes from sync chunk gens.
This is also a precursor to my next improvement to prioritize chunks
in front of the player (Frustum Priorization)
In most cases, this change won't benefit much. However, there
exists the possibility that your Chunk Task threads are all busy
doing super slow work such as converting chunks.
If this occurs, the main thread blocking tasks, even at highest priority,
has to wait for some thread to become available.
This change gives us a waiting thread used only for main thread blocking
tasks, as well as an increased thread priority level, so that the OS
will give priority to this thread over the other threads.
This is more about guarantees, and won't be any real performanc boost
to anyone who has low or fast activity on their chunk tasks anyways.
But not all of us force upgrade our worlds, and this can be a life saver.
also reordered some patches because multiple PR's were merged.
Forgot to flip the pending boolean back to false, causing it to copy
empty data on the next tick if nothing else triggered a load.
haven't managed to actually reproduce the crash others got, but did
verify that the bad copy was occurring erasing the data.
also fixed a bug with chunk load callback not executing before
another one was scheduled.
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing
CraftBukkit Changes:
183139d4 SPIGOT-5665: Improve loading spawn egg NBT
dec5df26 SPIGOT-5667: Can't add recipe without (vanilla) datapack
Spigot Changes:
ae72bf43 SPIGOT-5666: Customizable End City Seed
This can cause a nasty server lag the spawn chunks are not kept loaded
or they aren't finished loading yet, or if the world spawn radius is
larger than the keep loaded range.
By skipping this, we avoid potential for a large spike on server start.
Credit to Spotted for the idea
A lot of the new chunk system requires constant back and forth the main thread
to handle priority scheduling and ensuring conflicting tasks do not run at the
same time.
The issue is, these queues are only checked at either:
A) Sync Chunk Loads
B) End of Tick while sleeping
This results in generating chunks sitting waiting for a full tick to
complete before it will even start the next unit of work to do.
Additionally, this also delays loading of chunks until this same timing.
We will now periodically poll the chunk task queues throughout the tick,
looking for work to do.
We do this in a fair method that considers all worlds, not just the one being
ticked, so that each world can get 1 task procesed each before the next pass.
We also cap the throughput of these task processes to 1 per world per 0.1ms or
200 max per tick, to ensure that high volume of tasks do not overload the current
tick time.
In a view distance of 15, chunk loading performance was visually faster on the client.
Flying at high speed in spectator mode was able to keep up with chunk loading (as long as they are already generated)
Wiz mentioned that large WorldEdit operations cause light to run on
main thread. The queue was small, set to 5.. this bumps it to 20
but makes it configurable per-world.
The main risk of increasing this higher is during shutdown, some
queued light updates may be lost because mojang did not flush the
light engine on shutdown...
The queue size only puts a cap on max loss, doesn't solve that problem.
Don't touch this unless you know you have a problem and ok with the risk.
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing
Bukkit Changes:
7361a62e SPIGOT-5641: Add Block.getDrops(ItemStack, Entity)
1dc91b15 Add specific notes about what is not API
2b05ef88 #484: Allow statistics to be accessed for offline players
CraftBukkit Changes:
f7d6ad53 SPIGOT-5603: Use LootContext#lootingModifier in CraftLootTable
5838285d SPIGOT-5657: BlockPlaceEvent not cancelling for tripwire hooks
f325b9be SPIGOT-5641: Add Block.getDrops(ItemStack, Entity)
e25a2272 Fix some formatting in CraftHumanEntity
498540e0 Add Merchant slot delegate
b2de47d5 SPIGOT-5621: Add missing container types for opening InventoryView
aa3a2f27 #645: Allow statistics to be accessed for offline players
2122c0b1 #649: CraftBell should implement Bell
No longer clones visible chunks which is causing massive memory
allocation issues, likely the source of Humongous Objects on large servers.
Instead we just synchronize, clear and rebuild, reusing the same object buffers
as before with only 2 small objects created (FastIterator/MapEntry)
This should result in siginificant memory use reduction and improved GC behavior.
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing
Bukkit Changes:
122289ff Add FaceAttachable interface to handle Grindstone facing in common with Switches
a6db750e SPIGOT-5647: ZombieVillager entity should have getVillagerType()
CraftBukkit Changes:
bbe3d58e SPIGOT-5650: Lectern.setPage(int) causes a NullPointerException
3075579f Add FaceAttachable interface to handle Grindstone facing in common with Switches
95bd4238 SPIGOT-5647: ZombieVillager entity should have getVillagerType()
4d975ac3 SPIGOT-5617: setBlockData does not work when NotPlayEvent is called by redstone current
Try to use a faster chunk lookup for collision detection, and only
fall back to the original for nearby chunks.
The collision code takes an AABB and generates a cuboid of checks rather
than a cylinder, so at high velocity this can generate a lot of chunk checks.
Where I blocked movement did not consider velocity buildup, which I assume
then "unleashes" if something was really trying to push that entity, and moves
it a very large distance.
Additionally, this method was completely misnamed, as movementTick
is more "doLotsOfTickThings", and ended up breaking AI too, which the whole
point of temporary wake ups was to let AI run to trigger new immunity.
Also fixed numerous behavioral rules for Immunity to improve vanilla gameplay,
suchas bees that are angry or moving towards a flower or hive, any insentient
that is targetting any enemy (Accidently made it any player), and included flying
mobs such as phantoms by reducing the type check to insentient instead of Creature.
Also improved inWater immunity to consider if the mob is movable by water or not.
The entire reason the if statement exists is to only flush and print when done if flag is true
This avoids /save-all from hurting as much as it was before, such as from backup plugins.
CraftBukkit caused a regression here by making unloading chunks not
have a ticket added and returning unloaded future.
This caused entities who were killed in same tick their chunk is unloading
to not be able to be removed from the chunk.
This then results in dead entities lingering in the Chunk.
Combine that with a buggy detail of the previous implementation of
the Dupe UUID patch, then this was the likely source of the "Ghost entities"
If something calls register twice, and the world is ticking, it could be
enqueued to add twice.
Vs behavior of non ticking of just overwriting state.
We will now simply log a warning when this happens instead of crashing the server.
This was not applied correctly, and would completely blow up chunk entity
registration if this feature was turned off....
Additionally, change how the entities are removed to be more consistent with other code.
Surface some of the logs indicating there is a problem as we are having so many issues with
entities that we don't need to be surpressing logs like that.
1) Immunity no longer gives 20 tick immunity, each immunity check can
give its own tick value on how long it lasts, drastically cutting down on most to 0-1 ticks.
2) Fixed Villager Immunity to use proper 1.15 check for Breeding.
3) Fixed Water Mobs being 100% immune due to the inWater check...
4) Fixed flying mobs being 100% immune due to the !onGround check...
5) Made Insentient mobs only check for the hasTasks during immunity check window, not every single tick. this made them way more active than desired
- this puts behavior closer to inline with my original behavior in Spigot, but still does some checks to allow them temporary immunity, just not as much as before.
6) Inactive Entities would "inch" while trying to move, effectively getting nowhere. Now while an entity is inactive, it just won't even try to move.
- this saves us from the expensiveness of Entity movement 1 out of 20 ticks. Now they will only move while either active or triggered a true immunity.
upstream was doing a containsKey/get pattern, and always doing it at that.
that scenario is only even valid if were in the middle of a block place.
Optimize to check if the captured list even has values in it, and also to
just do a get call since the value can never be null.
We bandaided this logic on a case by case basis before with Loot Tables and Chests,
but that is not the correct fix.
Ultimately we just needed to clear out the captured tile entity list so that the
update calls did not process the tile entity that is no longer even valid.
This solution will ensure ALL placed tile entities data is ignored and not just
the few cases we previously had filters for.
While it wasn't really "broken" before, if plugins use NMS
(which they really should't be) and mess with entity management
themselves, and get it wrong, they could ultimately corrupt our
state expectations.
I've been unable to reproduce any issues locally, but these changes
are the result of me analyzing the code pretty deeply and seeing
about how to make it more durable to abnormal usage.
Any servers seeing oddities, please run with -Ddebug.entities=true
and send me any logs triggered.
We pretty much reverted most of Spigots changes here as ours is better.
I guess least Spigot is trying to get closer to us? but missed..
CraftBukkit Changes:
0bf75bbd SPIGOT-4686: Enchantment ordering nondeterministic
Spigot Changes:
f09662d0 Rebuild patches
this will help ensure that upstreams apply as cleanly as it does manually
the filtering makes the patches more fragile, but usually not an issue for small updates.
this will unfilter, apply the cleanest patches, then the next rebuild will
filter out unnecessary changes
Now decounts entities that got name tagged or persistent state changed.
Removes a bad CB check that ignored some persistent mobs persistence state.
Vanilla code explicitly only cares 'is this mob persistent?' and if
it is, it is not counted. CB should not of been checking its type state.
Fixes#1922
removes Mineshaft loading legacy as we had pre 1.13.2 to avoid managing
that very large data file from legacy systems.
Previous to 1.13.2 these data files were never loaded to begin with, so they
effectively do not contain valid/relevant data.
These files take a long time to convert on large worlds and crashes the server.
Additionally, cache the result of a file being missing so we don't keep spam checking it.
This code could get hit by many threads at once, causing multiple
chunk loads to convert legacy data, leading to all sorts of fun.
Additionally, go ahead and preload it async on world load.
Instead of allocating a buffer for every chunk compression, reuse
the same 64k sized buffer.
Also stopped doing dynamic compression levels. It wasn't helping enough.
This will improve memory usage and zlib performance of chunk compression.
Exposes a mutable array on items a player should keep on death.
This allows a cleaner method to implement "Keep certain items on death"
than how plugins currently do it in that it never removes them in first
place, so its safe if the player logs out/server is shutdown before respawn.
Example Usage: https://gist.github.com/aikar/5bb202de6057a051a950ce1f29feb0b4
getting lots of bug reports on light. just going to drop this even though
I really don't see how it could be the source due to my understanding of
the games light engine...
Fixes#1920
Use O(1) patterns for mapping conversions to improve plugin performance.
Optimize collections used by legacy plugin conversion
This should improve plugin performance across the board
on Paper for both Modern and Legacy Plugins.
Also log how long building those tables takes...
Some of these were wrong (scoreboard manager), others are counter to
everything everyone expects (Locations world being null, which wasnt ever safe EVER)
others are just too noisy.
Replace some with Contract to get rid of the nullability constraint and go back to
the old days of IDE not considering it strictly one way or the other.
Also, stop requiring annotations on package-private.
Introduces the next Developer Perk for Paper-API: Your plugin isn't yellow anymore.
Also fixed random dupe code in ensureServerConversions that got mistakenly set in the update.
use a better stack for managing timing tree relationships
remove unnecessary synchronization (i forgot HANDLERS is only touched on main)
this should hopefully resolve any data integrity concerns
turning this to true allows 1 player to deny other players the ability
to have mobs spawn. It really is a bad idea and reverts back to iffy
mob spawn limits where mob farms in the world stops natural spawns
from occuring.
Really recommend resorting to different solutions than ever using this config.
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing
Bukkit Changes:
451bd74e Use gson for decoding version command
CraftBukkit Changes:
51100db8 SPIGOT-4678: Fix placing signs in certain circumstances
This resolves the super common complaint about mobs not spawning.
This was ultimately a flaw in the vanilla count algorithim that allows
spawners and other misc mobs to count against the mob limit, which are
not bounded, and can prevent the entire world from spawning new.
I believe Bukkits changes around persistence may of actually made it
worse than vanilla.
This should fully solve all of the issues around it so that only natural
influences natural spawns.
Allows you to return the SpawnReason for why an Entity Spawned
Pre existing entities will return NATURAL if it was a non
persistenting Living Entity, SPAWNER for spawners,
or DEFAULT since data was not stored.
Mojang stored whitelist state in 2 places (Whitelist Object, PlayerList)
some things checked PlayerList, some checked object. This moves
everything to the Whitelist object.
I really don't know how this could break things with light, but wanted to
let it be an option so we can try turning it off to fix some light bug reports
Also adds a limit to stop sending Sign data to client after 500
signs per chunk to limit client lag.
Use -DPaper.excessiveSignsLimit=500 to configure that limit, or -1
to disable the limit and let your players be abused.
fixes#1878
now 80 chars counts multi sized code points the same so 80 chinese
characters would be allowed too.
Removed outbound limit as it doesn't solve the chunk oversize problem.
proper fix for chunk sending in another patch next.
we missed something in updating this patch and accidently left
hashCode as a 0 value. This caused a pretty nasty performance hit
for BlockStateEnum instances as they would be all a hash collision.
This update fixes that and also promotes the optimizations to cover
ALL cases of Block States, not just enums.
While mojang cached hashCode in 1.13, we can still just use a simple
large prime based hashCode that auto increments guaranteeing no overlap.
Additionally, replace all equals with an identity check.
modified clients can send abnormally large data from the client
to the server and it would get stored on the sign as sent.
the client can barely render around 16 characters as-is, but formatting
codes can get it to be more than 16 actual length.
Set a limit of 80 which should give an average of 16 characters 2
sets of legacy formatting codes which should be plenty for all uses.
This does not strip any existing data from the NBT as plugins
may use this for storing data out of the rendered area.
it only impacts data sent to and from the client.
Set -DPaper.maxSignLength=XX to change limit or -1 to disable
per feedback from @liach about a bug with vanillas previous logic
that we essentially just reverted to. complex recipes could run into
cases where a recipe that does have enough items can be unsatisfied
if the items are in certain orders, making them not truely shapeless.
this scenario doesn't occur in vanilla, but users can load custom recipes.
this improves the logic to do some sorting on the lists to improve
chances of matching the recipes harder ingredients before the easier
to satisfy ingredients.
the likelyhood of someone adding a recipe that still fails this is too
extreme to care about. i doubt mojangs implementation is perfect either.
we can improve this logic more later if we actually get bug reports on it.
Mojang implemented Shapeless different than Shaped
This made the Bukkit RecipeChoice API not work for Shapeless.
This reimplements vanilla logic using the same test logic as Shaped
Adds an event for when the server is going to destroy a current block,
potentially causing it to drop. This event can be cancelled to avoid
the block destruction, such as preventing signs from popping when
floating in the air.
This can replace many uses of BlockPhysicsEvent, as this will not fire
with the same velocity as BPE.
ConcurrentHashMap synchronizes on .computeIfAbsent even on hits,
so this does a .get(key) first, which most of our use should
be hits, and then falls back to the CHM computeIfAbsent for thread safe puts.
Also improve concurrency on handler and group collections to use a synchronized
list instead of an array deque for concurrency safety.
While these logs indicate something is 'off', we've had no reports
of visible 'issues' with entities, and no one is going to take the
time to debug this farther, so just getting rid of the logs so it's
like its never happening!
Ignorance is bliss.
This was a useless exception wrapper that ends up making
stack traces harder to read as well as the JVM cutting off
the important parts
Nothing catches this exception, so its safe to just get rid
of it and let the REAL exception bubble down
Upstream has released updates that appears to apply and compile correctly.
This update has only been PARTIALLY tested by PaperMC and as with ANY update, please do your own testing
I've tested basic region file saving as well as our oversized chunks approach.
Bukkit Changes:
e167e549 Clarify MerchantInventory#getSelectedRecipe.
3a1d5b8f Apply default permissions by registration order.
c64cc93f Make tags Keyed
ec037ed7 Added a method to get a list of tags
bfb6ef86 Introduce rotation methods to the Vector class
fc727372 Remove draft API from FluidLevelChangeEvent
CraftBukkit Changes:
6430d9c0 SPIGOT-4632: BlockState location is not fixed
14cd1688 Fix CraftInventoryMerchant#getSelectedRecipe if there is no active merchant recipe.
c24abab7 Load custom permissions after default permissions.
bc99dfe8 Make tags Keyed
6fce004f Added a method to get a list of tags
Spigot Changes:
e5e5c7c6 Allow Saving Large Chunks
e8d3881c Rebuild patches
Please test this build on a local TEST SERVER before sending to your live server!
PaperMC is not responsible for any data loss to your chunks.
-------------------------------------------------------------------
The Minecraft World Region File format has a hard cap of 1MB per chunk.
This is due to the fact that the header of the file format only allocates
a single byte for sector count, meaning a maximum of 256 sectors, at 4k per sector.
This limit can be reached fairly easily with books, resulting in the chunk being unable
to save to the world. Worse off, is that nothing printed when this occured, and silently
performed a chunk rollback on next load.
This leads to security risk with duplication and is being actively exploited.
This patch catches the too large scenario, falls back and moves any large Entity
or Tile Entity into a new compound, and this compound is saved into a different file.
On Chunk Load, we check for oversized status, and if so, we load the extra file and
merge the Entities and Tile Entities from the oversized chunk back into the level to
then be loaded as normal.
Once a chunk is returned back to normal size, the oversized flag will clear, and no
extra data file will exist.
This fix maintains compatability with all existing Anvil Region Format tools as it
does not alter the save format. They will just not know about the extra entities.
This fix also maintains compatability if someone switches server jars to one without
this fix, as the data will remain in the oversized file. Once the server returns
to a jar with this fix, the data will be restored.
If anything throwable is thrown, be sure its caught too.
Because I totally did new Error() out of javascript habit...
This ensures that chunk loading process cant get stuck due to uncaught errors
if a chunk load was cancelled after generating stage started
it would short circuit return with a null.
however this skipped the creation of the loadTask, which some code
would then invoke in requestChunk and trigger an NPE.
This then likely left an incomplete corrupt request in the chunk map
which then crashes servers.
It should fix these isseues
Fixes#1775Fixes#1743
Aikar