Devlog 2020
Weekly updates of the process on Jactorio
I spent the majority of this week cleaning up the Python API in preparation for entities. The main issue was the need to add every Prototype constructed to data_raw. This resulted in the need for large amounts of code to add some of the more derived Prototypes, consider the 2 functions below for adding a Tile:
Left: Prior to the new Python API | Right: Current Python API
This has been changed such that constructing a new prototype j.add(c.---) automatically adds it to data_raw, returning a reference to the added prototype to Python.
There is also the ability to chain property initialization .prop(val).prop2(val2) done with settlers that return a reference to itself. To read values, add a underscore _ in front of the property, the above to read becomes x = ._prop and y = ._prop2
This week's work was on the placement and pickup of entities. The placement logic is essentially complete, while the pickup still needs to return the item to the player's inventory.
https://i.imgur.com/P2eYrYV.gif
There are still a few little tweaks I will add in the future, such as giving transparency to the preview entity and having one large selection cursor for multi-tile entities instead of many small ones.
Added a recipe menu
There were also various improvements behind the scenes which should improve performance and allow for the implementation of features in the future. They will be mentioned as they are utilized in future weeks.
Added the fundamental inventory crafting mechanisms and gui features this week, see below:
https://i.imgur.com/lgJntOg.gif
It supports recursively crafting any ingredients needed for the final product. There is still the need to implement a queue next week so the crafting times are not instant. When an items is hovered, it will show the raw materials needed, craftable ingredients and missing ingredients.
The crafting queue unfinished from last week was implemented this week. In addition, the ability to extract resources was also added. This was a far more complex process than I imagined as it required tiles to store unique data, which was previously not possible. It is done however, and it can be seen in the video below.
https://i.imgur.com/1p5vKaI.gif
The core functionality of the player and its inventories has now been implemented using last week's addition of tiles holding unique data, which allowed for containers to be implemented. In further weeks, tiles holding unique data will be built upon to allow items to exist as entities in the world and later on belts.
https://i.imgur.com/EmQY5eL.gif
Introduced early belt logic:
https://i.imgur.com/50P3HV0.gif
My implementation was inspired by the various FFF posts by the Factorio team, notably: FFF-82, FFF-176
The items on the belts are designed to be as small as possible in terms of memory footprint. Knowing only their position and pointer to its Sprite. When updating belt logic for each chunk, each item's positions is matched against a std::map to determine if the item's direction needs to update (See below).
There are 4 locations outlined in yellow, when the item's world coordinates are matched against any of these 4 locations, their movement direction are updated.
The procedure above should provide an acceptable time complexity of O(logn) as the number of item direction update points grow and O(n) as item counts grow.
Completed the movement of items using transport line update locations, they can now move items across chunk boundaries, see below:
https://i.imgur.com/YVhXDds.gif
However, this is highly inefficient as each item must be iterated through and checked for updates. Thus, work began on grouping transport lines into "segments". See FFF-176 for more details.
Through this week's work, the transport lines are now grouped by segments moving in one direction. In addition to the far superior performance in terms of logic updates, this has allowed for additional features to be implemented:
https://i.imgur.com/A1hZOAB.gif
Items now have "collision", where they will maintain a set gap from the previous item. By not storing the item's positions as world coordinates, when rendering, the positions of these items can be freely interpolated, allowing for animations or curved corners which are purely aesthetic.
Fixed various bugs surrounding transport lines this week, mainly the issue of floating point inaccuracies which breaks the determinism of the transport lines.
https://i.imgur.com/yOaZHuJ.gif
To fix this, they have been replaced with a fixed precision decimal data type. Allowing for the introduction of stopping when the target transport line has stopped.
https://i.imgur.com/Z2VGeQi.gif
Transport lines also now support higher speeds and invisible items (for underground belts).
https://i.imgur.com/h99iSs9.gif
Added the ability for transport lines to feed onto a single side of a target transport line.
https://i.imgur.com/nNha2Ad.gif
Work has also begun this week on refactoring existing files for three goals:
- Completing various outlying tasks marked TODO
- Refactoring static namespace functions and variables within
jactorio::gameinto class instances - Enforcing a single uniform code style
Refactoring work from last week was completed, added the ability to render spritemap sections and rotating entities:
https://i.imgur.com/HOAyZXy.gif
Added the ability to manually place transport lines of arbitrary shapes in the game world. Prior to this, all transport lines were hard coded for demonstrations and testing.
https://i.imgur.com/s4M3lWK.gif
The placement logic still is not perfect as the items would occasionally get stuck in certain orientations, but the transport lines are almost complete!
Fixed a variety of transport line placement related bugs from last week. In addition to adding the ability to remove transport lines.
https://i.imgur.com/B41HE7a.gif
With transport lines being essentially complete except for optimizations which will be done later, this week's work focused on adding the first link in Factorio's supply chain: Mining drills.
https://i.imgur.com/kZxO2Yc.gif
They still have a few weeks of work remaining before they are complete. Being able to be rotated, removing resources after extracting a set amount and outputting onto transport belts.
Some more work on mining drills this week, which are now complete. Next up is fixing a few more newly discovered bugs with how transport lines behave.
https://i.imgur.com/2Zs9nTH.gif
Tidied up project source files and merged resource drills with master branch. See pr-5
With Resource Drills complete, next up on the list are inserters. However, since Factorio does not render inserters from spritesheets, I sought some help to create a 3D model and spritesheet to render from.
While that is underway, I improved the belts which are at the heart of every factory. Adding grouping of belts.
https://i.imgur.com/Sz4Aoo0.gif
Since each belt segment is updated at constant time regardless of the number of items, it is preferred to minimize the number of segments.
Completed the grouping logic for transport lines this week. Although work still needs to be done such that drills and future entities insert at the right location on the line.
A little side addition I saw the opportunity to add this week was animations. Most of the requisites were in place and I only needed to make a few modifications.
https://i.imgur.com/JpP1XWt.gif
It is by no means perfect right now, some animations run too fast and some frames are misaligned, but it will do for now.
Completed all grouping logic for transport lines. Some bugs still remain, for example it will not recognize that the player has placed a new line ahead and remain inactive, items are misaligned sometimes and a few others. I could continue working to perfect this, but I believe I should not spend all my time on this and work on something new for now.
Here is the same setup from May 8 but with all the features now implemented.
https://i.imgur.com/KexhMxE.gif
Next week will be something new and exciting, inserters!
Worked on upgrading internal data structures in preparation for inserters this week. This should make inserters far easier to implement.
Measured performance gains on Windows (Intel 6700k, GTX980)
- World draw 4ms, (-1ms)
- Memory usage 175Mb, (-75Mb)
Implemented detection of pickup and dropoff tiles for the inserter. See below, please excuse my inserter sprite for now:
Blue: Dropoff
Orange: Pickup, Red: Valid region on transport segment for pickup
Added inserter logic and bugfixes.
https://i.imgur.com/4I2Zu3J.gif
As you can see, inserters are far prettier now compared to last week, though they still require their arms and tweaks to the calculation of items valid for pickup on transport segments, this is a work in progress.
For a few months now, CI on MacOS fails with a segmentation fault. I have finally decided to properly investigate the matter by implementing stacktrace on crashes. It turned out to be an invalid OpenGL call, which occurred without an valid OpenGL context.
Not as much work this week and probably for the following few as I am busy with other affairs.
The pickup logic of the inserters were tweaked to be more accurate when picking up from transport segments.
In addition, work began on long armed inserters, those that pickup from > 1 tiles. This actually proved to be quite a considerable technical challenge since the pickup / dropoff locations are not neighboring the inserter. Thus the inserter does not receive an update event when these locations are placed or removed.
Corrected rendering issues surrounding entity animations this week. See the new remake of the setup from May 15 below:
https://i.imgur.com/iCtxIL6.gif
The rendering for the inserter arms is deferred to a later date, following weeks will see bugfixes to transport segments.
Fixed a variety of bugs surrounding transport segments this week, see pr-10.
Next week will see the beginning for the implementation of assembly machines
Spent part of the week cleaning up existing code before proceeding with assembly machines. Primarily removing global variables for parameters instead.
For the other half, the Gui was improved. Inventory slots can now be clicked in the margins between each slot, recipes can be searched and entity menus now display the corresponding entity's localized name.
https://i.imgur.com/aQWIGZL.gif
https://i.imgur.com/C3qKade.gif
Finished nearly all of the gui work and logic for assembly machines, only the progress bar remains.
https://i.imgur.com/nA7UUq6.gif
The crafting logic and inserter support should be done within the next 1 - 2 weeks.
Completed assembly machines! They will now craft the specified recipe and return the product.
https://i.imgur.com/wsLPAwc.gif
This can be used in conjunction with existing items to achieve increased automation.
https://i.imgur.com/EgWXx8C.gif
The old renderer served its original purpose, rendering a single layer of tiles. However this was not enough and as requirements grew, it had small hacks applied to it. The result of such hacks was duplicated code and poor performance.
A new greatly improved renderer replaces it this week, which utilizes a configurable number of threads to perform drawing in parallel. The threads utilize a z-axis added this week, allowing for unordered drawing which gave a reduction in draw calls and a millisecond reduction in frame times.
Now even when compiled in Debug configuration, a steady framerate of 60 at max zoom can be achieved compared to 20 in the past.
With the renderer improvements from last week, inserters can finally have arms! In addition, the inserters have also grown smarter, only picking up items it knows it can drop off.
https://i.imgur.com/67mDYeY.gif
Continuing inserter improvements from last week, inserters will only fill an assembly machine ingredient slot to 2 times the required amount. Continuing to make inserters easier to work with when handling items, they can now pick up from both lanes of a transport segment.
The final entity to gain improvements for now, mining drills will now hold an item until it can be outputted, allowing for better compressed transport lines. To encourage the player to expand around the world, drills are no longer infinite in the resources they output and now remove resources from the world.
For future weeks, work has started on one of the most important features in any game, that is saving and loading.
As of right now, the entire auto-generated world and be serialized. Support for entity data is ongoing and currently there is support for resource and container entities.
https://i.imgur.com/S2m9aOL.gif
As it turns out, serializing was a far from trivial task, since the serialization library Cereal does not handle raw pointers. In response, post deserialization work must be done to repoint all the pointers.
All prototypes and their respective data can be serialized alongside records of chunks requiring logic updates.
Player data still requires serialization support and bugs still persist during the deserialization process.
The entirety of the game state can now be stored and reloaded from disk.
https://i.imgur.com/P01MpWo.gif
Jactorio is now 1 year old!
In preparation for two new GUIs: main menu and hotbar, the current gui code was cleaned up. The progress involved removing magic numbers, splitting long calculations, extracting duplicate code and fixing the alignment of some widgets.
Added a main menu to allow world properties can be changed, currently only world seed.
Loading from save file is still in the works.
The main menu is now accessible from the start and in-game. In addition, loading and saving from disk is now also complete.
https://i.imgur.com/z3BJMrc.gif
Some bugfixes regarding the menus this week, they are now complete.
These was some confusion over what belonged in the namespace data. It is now defined as symbols which perform "External data handling". As such, prototypes were moved to a separate new namespace called proto.
Continuing from last week with namespace changes, here are the new namespaces:
- core: Generic functions, templates, algorithms
- data: External data handling
- proto: Prototypes
- game: Runtime, in-memory logic
- gui: GUI menus
- net: (future) Networking
- render: Rendering
No work was performed this week.
Some minor changes regarding file structure and file names
Happy Halloween!
No work was performed this week. But wait! I will return soon when I am free.
Merged work from previous weeks into the main branch.
Files are now consistent with each other in when accessing namespaces, as the introduction of using namespace jactorio; to shorten declarations was only introduced partway.
Next few weeks will be on adding configurable keybinds to finish off work on the main menu.
The GUI for changing key bindings is nearly complete, see below:
https://i.imgur.com/9zc0Lww.gif
It still needs additional work to display modifier keys and localization for the key actions; all of which should be done next week.
Keys can be bound with modifiers, key actions are localized, custom keybinds now get saved to file and are loaded on game start.
To avoid duplicate code, splitters will use the same connection logic as transport belts. Thus, work begun on extracting conveyor connection logic to be reusable for splitters.
Finished extracting logic for connecting conveyors, now on logic for disconnecting conveyors.
Since the logic for disconnecting conveyors must ungroup conveyors, I had to extract the logic for grouping conveyors first. Not all that complicated, I put in a bit of work here and there when I have the time.
Finally extracted and cleaned up the mess that was conveyor placement logic, now just a few function calls instead of a thousand line file full of odd templates and functions with no clear purpose.