Jump to content

[1.10.2] Frame-based recording and synchronization issues


horsewithnoname

Recommended Posts

Hello there!

 

I'm working on a mod that allows recording player movement and some actions (like block interaction, dropping items, etc.). In previous versions, the record and playback of player movement and actions, was pretty primitive, it simply recorded position, rotation, etc. in separate thread and played back those actions also in separate thread. For the next release, I wanted to update recording code to the next level (so it would be very smooth and Minema compatible). I did implemented smooth recording, but I experience tick desynchroinzation issues sometime.

 

Right now, the recording code works following way (source code if somebody needs more than an explanation):

 

1. Player notifies server that he wants to record himself (via command or interacting with actor mob)

2. Server starts recording and sends a message to the client that recording has started

3. Server records actions (pushed via event handler on the server-side) and client records position, rotation, motion, fall distance and some booleans (onGround, isSneaking, etc.)

4. When player asks the server to stop recording (via command or interacting with actor mob), server asks player for the recorded frames, and puts recorded actions on hold until client frames will arrive

5. When client frames are arrived, server adjusts the length of the action list and saves frames and actions to the file

 

There are no issues with recording, but with playback, I experience tick desynchronization issue. I don't know why it happens, but sometimes, client update loop can't keep up with server loop, I guess, and in this case it makes playback look really awful (especially while flying elytra).

 

Playback code works similar to recording code, but it's more complicated:

 

1. When actor is starting playing back (requested by player), actor being played from recorded record on the server

2. On the client, however, recording might not exist, so the server will send actor tracking clients record, if they don't have it, while giving an actor empty playback which will just run ticks (in order to keep up with the server)

3. When the record is finally arrived on the client, it being injected into the empty playback which will then start played back on the client every tick

4. When playback is finished, server notifies the players that an actor stopped playing

5. Clients will simply remove the playback from the actor and next time, they would be able to use the same record for playback

 

tl;dr. Yes, it's a pretty complicated system, but I'm not asking to rewrite it for me. I described how it works, if you needed that info, to answer my question/help me with my problem.

 

So now is the question time. How can I resolve this issue with tick desynchronization? I see only two options:

 

1. Rewrite playback code so it will be played only on server (i.e. every tick server going to send packets with position, rotation, motion, etc.)

2. Every N ticks, send to the client the number of ticks server has

 

I implemented currently 2. as a workaround, but I feel that with slow connection the arrived tick will probably be outdated by the time it arrived. But with 1., I fear it will create really dense server load when there are lots of players and lots of actors being played back.

 

What will be the optimal solution?

 

Thanks for attention! (sorry for the wall of text) :)

Blockbuster – simple machinimas and cinematics for Minecraft
Link to comment
Share on other sites

Hi

 

I think your option 1 is the best bet for live action stuff and probably the most resistant to network lag.

 

However if you're actually aiming for a machinima rendering that doesn't need to be in real time, you might be better off locking the ticks between client and server eg with a handshaking, to ensure there is never any desynchronisation. 

 

I don't think the update-tick-count resynchronise is likely to help much with lag / judder.  If you are just dropping frames now and again, it might work if you can assume a fairly constant network latency that you can correct for.

You could perhaps improve it by buffering ahead with timestamps, for example the server runs its ticks strictly according to the timestamps compared to the system clock, and the same on the client;  but I think this would still rather sensitive to network delay.

 

I think the best answer will probably depend on exactly what you want to achieve with the playback and probably the best way to find the answer is to code it a few different ways and stress test it.

 

-TGG

 

 

 

Link to comment
Share on other sites

Thanks for the answer TGG, by the way, you have really nice tutorials from which I learned a lot :)

 

However if you're actually aiming for a machinima rendering that doesn't need to be in real time, you might be better off locking the ticks between client and server eg with a handshaking, to ensure there is never any desynchronisation.

 

Well, for now, yes, I'm aiming for machinima rendering. With machinima rendering, it's actually pretty good, I haven't noticed any frame desynchronization since Minema does the synchronzation work for me, but for live stuff I'm not sure about it.

 

In the future, I would like to expand this recording/playback code to make cinematic stuff like where there are NPC's that do some pre-recorded stuff like in Borderlands 2 or Skyrim (just from the top of my head). I do not aim at multiplayer servers, though.

 

Your offer seems a little bit less expensive than option 1, and probably will have some issues during the lag (probably, although lag is the problem itself).

 

I don't think the update-tick-count resynchronise is likely to help much with lag / judder.  If you are just dropping frames now and again, it might work if you can assume a fairly constant network latency that you can correct for.

 

You're correct. Depending on the network latency it may drastically affect the playback. I tested everything only in single-player. By the way, I'm not aiming my mod toward public servers.

 

I think the best answer will probably depend on exactly what you want to achieve with the playback and probably the best way to find the answer is to code it a few different ways and stress test it.

 

Ok, I'll try different ways in the future, thank you :)

 

P.S.: What does stress test means?

Blockbuster – simple machinimas and cinematics for Minecraft
Link to comment
Share on other sites

Hi

 

> Thanks for the answer TGG, by the way, you have really nice tutorials from which I learned a lot :)

Keen, glad you found them useful :)

 

What I mean by 'stress test' is testing your code with difficult conditions - like high latency, complicated animation, high server CPU load, that sort of thing.

 

Let me know how it goes?  It sounds very interesting.

 

-TGG

 

Link to comment
Share on other sites

What I mean by 'stress test' is testing your code with difficult conditions - like high latency, complicated animation, high server CPU load, that sort of thing.

 

Oh, I get it now, thanks :)

 

Let me know how it goes?  It sounds very interesting.

 

Of course :) I think I'll rewrite in the next update to the option 1.

Blockbuster – simple machinimas and cinematics for Minecraft
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Announcements



  • Recently Browsing

    • No registered users viewing this page.
  • Posts

    • I have made a post shader with all of it's files (in post and program) being finished, it's just an edited creeper post shader for now. How do I load it ingame?  Minecraft.getInstance().gameRenderer.loadEffect(new ResourceLocation("thedefused:shaders/post/creep.json")); This just gives an error due to it trying to find program in minecraft:shaders/program However, I have seen multiple mods use post shaders. how?
    • Hello. I've been having a problem when launching minecraft forge. It just doesn't open the game, and leaves me with this "(exit code 1)" error. Both regular and optifine versions of minecraft launch just fine, tried both with 1.18.2 and 1.20.1. I can assure that my drivers are updated so that can't be it, and i've tried using Java 17, 18 and 21 to no avail. Even with no mods installed, the thing won't launch. I'll leave the log here, although it's in spanish: https://jmp.sh/s/FPqGBSi30fzKJDt2M1gc My specs are this: Ryzen 3 4100 || Radeon R9 280x || 16gb ram || Windows 10 I'd appreciate any help, thank you in advance.
    • Hey, Me and my friends decided to start up a Server with "a few" mods, the last few days everything went well we used all the items we wanted. Now our Game crashes the moment we touch a Lava Bucket inside our Inventory. It just instantly closes and gives me an "Alc Cleanup"  Crash screen (Using GDLauncher). I honestly dont have a clue how to resolve this error. If anyone could help id really appreciate it, I speak German and Englisch so you can choose whatever you speak more fluently. Thanks in Advance. Plus I dont know how to link my Crash Report help for that would be nice too whoops
    • I hosted a minecraft server and I modded it, and there is always an error on the console which closes the server. If someone knows how to repair it, it would be amazing. Thank you. I paste the crash report down here: ---- Minecraft Crash Report ---- WARNING: coremods are present:   llibrary (llibrary-core-1.0.11-1.12.2.jar)   WolfArmorCore (WolfArmorAndStorage-1.12.2-3.8.0-universal-signed.jar)   AstralCore (astralsorcery-1.12.2-1.10.27.jar)   CreativePatchingLoader (CreativeCore_v1.10.71_mc1.12.2.jar)   SecurityCraftLoadingPlugin ([1.12.2] SecurityCraft v1.9.8.jar)   ForgelinPlugin (Forgelin-1.8.4.jar)   midnight (themidnight-0.3.5.jar)   FutureMC (Future-MC-0.2.19.jar)   SpartanWeaponry-MixinLoader (SpartanWeaponry-1.12.2-1.5.3.jar)   Backpacked (backpacked-1.4.3-1.12.2.jar)   LoadingPlugin (Reskillable-1.12.2-1.13.0.jar)   LoadingPlugin (Bloodmoon-MC1.12.2-1.5.3.jar) Contact their authors BEFORE contacting forge // There are four lights! Time: 3/28/24 12:17 PM Description: Exception in server tick loop net.minecraftforge.fml.common.LoaderException: java.lang.NoClassDefFoundError: net/minecraft/client/multiplayer/WorldClient     at net.minecraftforge.fml.common.AutomaticEventSubscriber.inject(AutomaticEventSubscriber.java:89)     at net.minecraftforge.fml.common.FMLModContainer.constructMod(FMLModContainer.java:612)     at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)     at java.lang.reflect.Method.invoke(Method.java:498)     at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:91)     at com.google.common.eventbus.Subscriber$SynchronizedSubscriber.invokeSubscriberMethod(Subscriber.java:150)     at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:76)     at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:399)     at com.google.common.eventbus.Subscriber.dispatchEvent(Subscriber.java:71)     at com.google.common.eventbus.Dispatcher$PerThreadQueuedDispatcher.dispatch(Dispatcher.java:116)     at com.google.common.eventbus.EventBus.post(EventBus.java:217)     at net.minecraftforge.fml.common.LoadController.sendEventToModContainer(LoadController.java:219)     at net.minecraftforge.fml.common.LoadController.propogateStateMessage(LoadController.java:197)     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)     at java.lang.reflect.Method.invoke(Method.java:498)     at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:91)     at com.google.common.eventbus.Subscriber$SynchronizedSubscriber.invokeSubscriberMethod(Subscriber.java:150)     at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:76)     at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:399)     at com.google.common.eventbus.Subscriber.dispatchEvent(Subscriber.java:71)     at com.google.common.eventbus.Dispatcher$PerThreadQueuedDispatcher.dispatch(Dispatcher.java:116)     at com.google.common.eventbus.EventBus.post(EventBus.java:217)     at net.minecraftforge.fml.common.LoadController.distributeStateMessage(LoadController.java:136)     at net.minecraftforge.fml.common.Loader.loadMods(Loader.java:595)     at net.minecraftforge.fml.server.FMLServerHandler.beginServerLoading(FMLServerHandler.java:98)     at net.minecraftforge.fml.common.FMLCommonHandler.onServerStart(FMLCommonHandler.java:333)     at net.minecraft.server.dedicated.DedicatedServer.func_71197_b(DedicatedServer.java:125)     at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:486)     at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.NoClassDefFoundError: net/minecraft/client/multiplayer/WorldClient     at java.lang.Class.getDeclaredMethods0(Native Method)     at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)     at java.lang.Class.privateGetPublicMethods(Class.java:2902)     at java.lang.Class.getMethods(Class.java:1615)     at net.minecraftforge.fml.common.eventhandler.EventBus.register(EventBus.java:82)     at net.minecraftforge.fml.common.AutomaticEventSubscriber.inject(AutomaticEventSubscriber.java:82)     ... 31 more Caused by: java.lang.ClassNotFoundException: net.minecraft.client.multiplayer.WorldClient     at net.minecraft.launchwrapper.LaunchClassLoader.findClass(LaunchClassLoader.java:191)     at java.lang.ClassLoader.loadClass(ClassLoader.java:418)     at java.lang.ClassLoader.loadClass(ClassLoader.java:351)     ... 37 more Caused by: net.minecraftforge.fml.common.asm.ASMTransformerWrapper$TransformerException: Exception in class transformer net.minecraftforge.fml.common.asm.transformers.SideTransformer@4e558728 from coremod FMLCorePlugin     at net.minecraftforge.fml.common.asm.ASMTransformerWrapper$TransformerWrapper.transform(ASMTransformerWrapper.java:260)     at net.minecraft.launchwrapper.LaunchClassLoader.runTransformers(LaunchClassLoader.java:279)     at net.minecraft.launchwrapper.LaunchClassLoader.findClass(LaunchClassLoader.java:176)     ... 39 more Caused by: java.lang.RuntimeException: Attempted to load class bsb for invalid side SERVER     at net.minecraftforge.fml.common.asm.transformers.SideTransformer.transform(SideTransformer.java:62)     at net.minecraftforge.fml.common.asm.ASMTransformerWrapper$TransformerWrapper.transform(ASMTransformerWrapper.java:256)     ... 41 more A detailed walkthrough of the error, its code path and all known details is as follows: --------------------------------------------------------------------------------------- -- System Details -- Details:     Minecraft Version: 1.12.2     Operating System: Linux (amd64) version 5.10.0-28-cloud-amd64     Java Version: 1.8.0_382, Temurin     Java VM Version: OpenJDK 64-Bit Server VM (mixed mode), Temurin     Memory: 948745536 bytes (904 MB) / 1564999680 bytes (1492 MB) up to 7635730432 bytes (7282 MB)     JVM Flags: 2 total; -Xmx8192M -Xms256M     IntCache: cache: 0, tcache: 0, allocated: 0, tallocated: 0     FML: MCP 9.42 Powered by Forge 14.23.5.2860 63 mods loaded, 63 mods active     States: 'U' = Unloaded 'L' = Loaded 'C' = Constructed 'H' = Pre-initialized 'I' = Initialized 'J' = Post-initialized 'A' = Available 'D' = Disabled 'E' = Errored     | State | ID                 | Version                 | Source                                                | Signature                                |     |:----- |:------------------ |:----------------------- |:----------------------------------------------------- |:---------------------------------------- |     | LC    | minecraft          | 1.12.2                  | minecraft.jar                                         | None                                     |     | LC    | mcp                | 9.42                    | minecraft.jar                                         | None                                     |     | LC    | FML                | 8.0.99.99               | forge-1.12.2-14.23.5.2860.jar                         | e3c3d50c7c986df74c645c0ac54639741c90a557 |     | LC    | forge              | 14.23.5.2860            | forge-1.12.2-14.23.5.2860.jar                         | e3c3d50c7c986df74c645c0ac54639741c90a557 |     | LC    | creativecoredummy  | 1.0.0                   | minecraft.jar                                         | None                                     |     | LC    | backpacked         | 1.4.2                   | backpacked-1.4.3-1.12.2.jar                           | None                                     |     | LC    | itemblacklist      | 1.4.3                   | ItemBlacklist-1.4.3.jar                               | None                                     |     | LC    | securitycraft      | v1.9.8                  | [1.12.2] SecurityCraft v1.9.8.jar                     | None                                     |     | LC    | aiimprovements     | 0.0.1.3                 | AIImprovements-1.12-0.0.1b3.jar                       | None                                     |     | LC    | jei                | 4.16.1.301              | jei_1.12.2-4.16.1.301.jar                             | None                                     |     | LC    | appleskin          | 1.0.14                  | AppleSkin-mc1.12-1.0.14.jar                           | None                                     |     | LC    | baubles            | 1.5.2                   | Baubles-1.12-1.5.2.jar                                | None                                     |     | LC    | astralsorcery      | 1.10.27                 | astralsorcery-1.12.2-1.10.27.jar                      | a0f0b759d895c15ceb3e3bcb5f3c2db7c582edf0 |     | LC    | attributefix       | 1.0.12                  | AttributeFix-Forge-1.12.2-1.0.12.jar                  | None                                     |     | LC    | atum               | 2.0.20                  | Atum-1.12.2-2.0.20.jar                                | None                                     |     | LC    | bloodmoon          | 1.5.3                   | Bloodmoon-MC1.12.2-1.5.3.jar                          | d72e0dd57935b3e9476212aea0c0df352dd76291 |     | LC    | forgelin           | 1.8.4                   | Forgelin-1.8.4.jar                                    | None                                     |     | LC    | bountiful          | 2.2.2                   | Bountiful-2.2.2.jar                                   | None                                     |     | LC    | camera             | 1.0.10                  | camera-1.0.10.jar                                     | None                                     |     | LC    | chisel             | MC1.12.2-1.0.2.45       | Chisel-MC1.12.2-1.0.2.45.jar                          | None                                     |     | LC    | collective         | 3.0                     | collective-1.12.2-3.0.jar                             | None                                     |     | LC    | reskillable        | 1.12.2-1.13.0           | Reskillable-1.12.2-1.13.0.jar                         | None                                     |     | LC    | compatskills       | 1.12.2-1.17.0           | CompatSkills-1.12.2-1.17.0.jar                        | None                                     |     | LC    | creativecore       | 1.10.0                  | CreativeCore_v1.10.71_mc1.12.2.jar                    | None                                     |     | LC    | customnpcs         | 1.12                    | CustomNPCs_1.12.2-(05Jul20).jar                       | None                                     |     | LC    | darknesslib        | 1.1.2                   | DarknessLib-1.12.2-1.1.2.jar                          | 220f10d3a93b3ff5fbaa7434cc629d863d6751b9 |     | LC    | dungeonsmod        | @VERSION@               | DungeonsMod-1.12.2-1.0.8.jar                          | None                                     |     | LC    | enhancedvisuals    | 1.3.0                   | EnhancedVisuals_v1.4.4_mc1.12.2.jar                   | None                                     |     | LC    | extrautils2        | 1.0                     | extrautils2-1.12-1.9.9.jar                            | None                                     |     | LC    | futuremc           | 0.2.6                   | Future-MC-0.2.19.jar                                  | None                                     |     | LC    | geckolib3          | 3.0.30                  | geckolib-forge-1.12.2-3.0.31.jar                      | None                                     |     | LC    | gottschcore        | 1.15.1                  | GottschCore-mc1.12.2-f14.23.5.2859-v1.15.1.jar        | None                                     |     | LC    | hardcorerevival    | 1.2.0                   | HardcoreRevival_1.12.2-1.2.0.jar                      | None                                     |     | LC    | waila              | 1.8.26                  | Hwyla-1.8.26-B41_1.12.2.jar                           | None                                     |     | LE    | imsm               | 1.12                    | Instant Massive Structures Mod 1.12.2.jar             | None                                     |     | L     | journeymap         | 1.12.2-5.7.1p2          | journeymap-1.12.2-5.7.1p2.jar                         | None                                     |     | L     | mobsunscreen       | @version@               | mobsunscreen-1.12.2-3.1.5.jar                         | None                                     |     | L     | morpheus           | 1.12.2-3.5.106          | Morpheus-1.12.2-3.5.106.jar                           | None                                     |     | L     | llibrary           | 1.7.20                  | llibrary-1.7.20-1.12.2.jar                            | None                                     |     | L     | mowziesmobs        | 1.5.8                   | mowziesmobs-1.5.8.jar                                 | None                                     |     | L     | nocubessrparmory   | 3.0.0                   | NoCubes_SRP_Combat_Addon_3.0.0.jar                    | None                                     |     | L     | nocubessrpnests    | 3.0.0                   | NoCubes_SRP_Nests_Addon_3.0.0.jar                     | None                                     |     | L     | nocubessrpsurvival | 3.0.0                   | NoCubes_SRP_Survival_Addon_3.0.0.jar                  | None                                     |     | L     | nocubesrptweaks    | V4.1                    | nocubesrptweaks-V4.1.jar                              | None                                     |     | L     | patchouli          | 1.0-23.6                | Patchouli-1.0-23.6.jar                                | None                                     |     | L     | artifacts          | 1.1.2                   | RLArtifacts-1.1.2.jar                                 | None                                     |     | L     | rsgauges           | 1.2.8                   | rsgauges-1.12.2-1.2.8.jar                             | None                                     |     | L     | rustic             | 1.1.7                   | rustic-1.1.7.jar                                      | None                                     |     | L     | silentlib          | 3.0.13                  | SilentLib-1.12.2-3.0.14+168.jar                       | None                                     |     | L     | scalinghealth      | 1.3.37                  | ScalingHealth-1.12.2-1.3.42+147.jar                   | None                                     |     | L     | lteleporters       | 1.12.2-3.0.2            | simpleteleporters-1.12.2-3.0.2.jar                    | None                                     |     | L     | spartanshields     | 1.5.5                   | SpartanShields-1.12.2-1.5.5.jar                       | None                                     |     | L     | spartanweaponry    | 1.5.3                   | SpartanWeaponry-1.12.2-1.5.3.jar                      | None                                     |     | L     | srparasites        | 1.9.18                  | SRParasites-1.12.2v1.9.18.jar                         | None                                     |     | L     | treasure2          | 2.2.0                   | Treasure2-mc1.12.2-f14.23.5.2859-v2.2.1.jar           | None                                     |     | L     | treeharvester      | 4.0                     | treeharvester_1.12.2-4.0.jar                          | None                                     |     | L     | twilightforest     | 3.11.1021               | twilightforest-1.12.2-3.11.1021-universal.jar         | None                                     |     | L     | variedcommodities  | 1.12.2                  | VariedCommodities_1.12.2-(31Mar23).jar                | None                                     |     | L     | voicechat          | 1.12.2-2.4.32           | voicechat-forge-1.12.2-2.4.32.jar                     | None                                     |     | L     | wolfarmor          | 3.8.0                   | WolfArmorAndStorage-1.12.2-3.8.0-universal-signed.jar | None                                     |     | L     | worldborder        | 2.3                     | worldborder_1.12.2-2.3.jar                            | None                                     |     | L     | midnight           | 0.3.5                   | themidnight-0.3.5.jar                                 | None                                     |     | L     | structurize        | 1.12.2-0.10.277-RELEASE | structurize-1.12.2-0.10.277-RELEASE.jar               | None                                     |     Loaded coremods (and transformers):  llibrary (llibrary-core-1.0.11-1.12.2.jar)   net.ilexiconn.llibrary.server.core.plugin.LLibraryTransformer   net.ilexiconn.llibrary.server.core.patcher.LLibraryRuntimePatcher WolfArmorCore (WolfArmorAndStorage-1.12.2-3.8.0-universal-signed.jar)    AstralCore (astralsorcery-1.12.2-1.10.27.jar)    CreativePatchingLoader (CreativeCore_v1.10.71_mc1.12.2.jar)    SecurityCraftLoadingPlugin ([1.12.2] SecurityCraft v1.9.8.jar)    ForgelinPlugin (Forgelin-1.8.4.jar)    midnight (themidnight-0.3.5.jar)   com.mushroom.midnight.core.transformer.MidnightClassTransformer FutureMC (Future-MC-0.2.19.jar)   thedarkcolour.futuremc.asm.CoreTransformer SpartanWeaponry-MixinLoader (SpartanWeaponry-1.12.2-1.5.3.jar)    Backpacked (backpacked-1.4.3-1.12.2.jar)   com.mrcrayfish.backpacked.asm.BackpackedTransformer LoadingPlugin (Reskillable-1.12.2-1.13.0.jar)   codersafterdark.reskillable.base.asm.ClassTransformer LoadingPlugin (Bloodmoon-MC1.12.2-1.5.3.jar)   lumien.bloodmoon.asm.ClassTransformer     Profiler Position: N/A (disabled)     Is Modded: Definitely; Server brand changed to 'fml,forge'     Type: Dedicated Server (map_server.txt)
    • When i add mods like falling leaves, visuality and kappas shaders, even if i restart Minecraft they dont show up in the mods menu and they dont work
  • Topics

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.