Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[core] new-pub-sub-matching-2 #1653

Merged
merged 7 commits into from
Jul 17, 2024
Merged

[core] new-pub-sub-matching-2 #1653

merged 7 commits into from
Jul 17, 2024

Conversation

KerstinKeller
Copy link
Contributor

@KerstinKeller KerstinKeller commented Jul 10, 2024 •
edited by rex-schilasky
Loading

Description

  • layer states (enabling) exchanged between pub/sub
  • layer activation based on layer priority lists for local and remote connections

@KerstinKeller @rex-schilasky
[core] New pub sub Matching
422fc85 
Co-authored-by: Rex Schilasky <[email protected]>
github-actions[bot]
github-actions bot reviewed Jul 10, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

ecal/core/src/readwrite/ecal_writer.cpp Outdated
TLayer::eTransportLayer CDataWriter::DetermineTransportLayer2Start(const std::vector<eTLayerType>& enabled_pub_layer_, const std::vector<eTLayerType>& enabled_sub_layer_, bool same_host_)
{
// determine the priority list to use
Publisher::Configuration::LayerPriorityVector& layer_priority_vector = same_host_ ? m_config.layer_priority_local : m_config.layer_priority_remote;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: variable 'layer_priority_vector' of type 'Publisher::Configuration::LayerPriorityVector &' (aka 'vectorTLayer::eTransportLayer &') can be declared 'const' [misc-const-correctness]

Suggested change
Publisher::Configuration::LayerPriorityVector& layer_priority_vector = same_host_ ? m_config.layer_priority_local : m_config.layer_priority_remote;
Publisher::Configuration::LayerPriorityVector const& layer_priority_vector = same_host_ ? m_config.layer_priority_local : m_config.layer_priority_remote;

@rex-schilasky rex-schilasky added the cherry-pick-to-NONE Don't cherry-pick these changes label Jul 10, 2024
@rex-schilasky rex-schilasky changed the title [core] New pub sub Matching [core] new-pub-sub-matching-2 Jul 10, 2024
@rex-schilasky rex-schilasky self-requested a review July 17, 2024 14:37
rex-schilasky
rex-schilasky approved these changes Jul 17, 2024
View reviewed changes
Copy link
Contributor

@rex-schilasky rex-schilasky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Team review done (as best as possible)

@rex-schilasky rex-schilasky merged commit 4659192 into master Jul 17, 2024
20 checks passed
@rex-schilasky rex-schilasky deleted the feature/new-pub-sub-matching-2 branch July 17, 2024 16:01
ashariff-11 added a commit to ashariff-11/ecal that referenced this pull request Aug 28, 2024
@ashariff-11
Squashed commit of the following:
427ad25 
commit e13cadb
Author: Peguen <[email protected]>
Date:   Fri Aug 23 14:30:08 2024 +0200

    [config] Move host_group_name logic (eclipse-ecal#1720)

    * Removed HostName in registration config, added usage in attribute builder.

ecal/tests/cpp/pubsub_proto_test/src/commit a410305
Author: KerstinKeller <[email protected]>
Date:   Fri Aug 23 14:27:54 2024 +0200

    [core] Bugfix: Publishers / Subscribers need to use global configuration, when no configuration is provided. (eclipse-ecal#1721)

commit 4f8bad0
Author: Peguen <[email protected]>
Date:   Thu Aug 22 12:56:37 2024 +0200

    [config] Removed quoteStrings and unused variable in config test. (eclipse-ecal#1718)

commit bfea308
Author: KerstinKeller <[email protected]>
Date:   Tue Aug 20 16:17:23 2024 +0200

    [core] remove monitoring timeout (registration timeout is only valid timeout). Set registration timeout to 10 seconds. (eclipse-ecal#1714)

commit bda08a6
Author: Rex Schilasky <[email protected]>
Date:   Tue Aug 20 14:37:30 2024 +0200

    python monitoring dictionary enhanced (still not complete) (eclipse-ecal#1715)

commit 5779c07
Author: Rex Schilasky <[email protected]>
Date:   Tue Aug 20 12:00:45 2024 +0200

    [sample] sample massive_pub_sub extended (eclipse-ecal#1713)

commit d4fd7d6
Author: KerstinKeller <[email protected]>
Date:   Tue Aug 20 11:49:19 2024 +0200

    [core] Rewrite GetTopicsParallel test to be more robust. (eclipse-ecal#1706)

commit 9db33c8
Author: Peguen <[email protected]>
Date:   Tue Aug 20 10:05:06 2024 +0200

    [config] Monitoring uses SAttributes instead of config object (eclipse-ecal#1710)

commit 998b3ff
Author: Peguen <[email protected]>
Date:   Tue Aug 20 09:01:06 2024 +0200

    [GH] Hotfix macos build & dependency (Qt5, capnp, python) brew installation (eclipse-ecal#1711)

    * Python, Qt5, capnp installation via brew (all latest)
    * Python packages installation with parameter --break-system-packages
    * Added Qt5 installation path to the CMAKE_PREFIX_PATH in installation

commit c974e6a
Author: KerstinKeller <[email protected]>
Date:   Tue Aug 20 08:44:53 2024 +0200

    [core] eCAL sample completely internal, only SEntityId public. (eclipse-ecal#1712)

commit 6a44dad
Author: Rex Schilasky <[email protected]>
Date:   Mon Aug 19 16:49:44 2024 +0200

    [core] feature/id-based-descgate (eclipse-ecal#1708)

    descgate API changed to use separate GetEntityIDs() + GetEntityInfo(id) functions instead of single GetEntities() variants

commit fe1fe22
Author: Peguen <[email protected]>
Date:   Mon Aug 19 15:39:17 2024 +0200

    [config] Added config object to registration. (eclipse-ecal#1709)

    * Added Config object to registration.

commit f80ce01
Author: KerstinKeller <[email protected]>
Date:   Tue Aug 13 15:14:56 2024 +0200

    [core] warnings and build issue fixes (eclipse-ecal#1707)

commit 938c891
Author: KerstinKeller <[email protected]>
Date:   Mon Aug 12 12:54:52 2024 +0200

    [core] unregistration of timed out samples in one central place (instead of all over the place). (eclipse-ecal#1675)

    Modifies also CExpirationMap interface to return keys and values of expired items.

commit 09d2cc8
Author: Peguen <[email protected]>
Date:   Thu Aug 8 12:11:14 2024 +0200

    [config] Added config object to monitoring initialization. (eclipse-ecal#1699)

    * Added config object to monitoring initialization

commit ab0870c
Author: Rex Schilasky <[email protected]>
Date:   Thu Aug 8 11:19:31 2024 +0200

    [core] registration specific functions moved from Util:: to Registration:: (eclipse-ecal#1700)

    * registration specific functions moved from Util:: to Registration::

commit 2b2e4d7
Author: Rex Schilasky <[email protected]>
Date:   Wed Aug 7 13:39:01 2024 +0200

    [core] enable loopback for monitoring functionality (eclipse-ecal#1698)

commit 0277978
Author: KerstinKeller <[email protected]>
Date:   Wed Aug 7 09:52:22 2024 +0200

    [core] Remove SPublicationInfo / SSubscriptionInfo type. (eclipse-ecal#1695)

    Instead Registration::SampleIdentifier are used in datawriter / datareader.

commit 5a4300b
Author: KerstinKeller <[email protected]>
Date:   Tue Aug 6 15:34:24 2024 +0200

    [core] Registration::SampleIdentifier and partial sample serialization

    eCAL Registration samples have a command type, and some member types.
    Now the serialization is performed only partially, according to the command type.

    Additionally, every sample now has a unique ID, marking the entity which has produced this sample.
    This info will be unique within the whole eCAL Ecosystem.

commit 6886ee4
Author: Rex Schilasky <[email protected]>
Date:   Tue Aug 6 15:06:48 2024 +0200

    [core] remove callbacks before stopping tcp protocol layer (eclipse-ecal#1693)

commit 9e8ecbb
Author: Peguen <[email protected]>
Date:   Mon Aug 5 08:22:09 2024 +0200

    [config] Updated documentation to new configuration structure (eclipse-ecal#1690)

    Only doc changes. New yaml references and config settings in code snippets.

    Issue link for current documentation checkup:
    eclipse-ecal#1672

commit e6dbb82
Author: Rex Schilasky <[email protected]>
Date:   Fri Aug 2 16:14:31 2024 +0200

    [core] missing include (std::find) (eclipse-ecal#1691)

commit a5d9eb4
Author: Peguen <[email protected]>
Date:   Fri Aug 2 12:12:44 2024 +0200

    [config] configuration yaml generation (eclipse-ecal#1680)

    * Config initialization happens now directly in structure.

    * Removed eCAL defs except for non configurable defs.

    * Adapted subscriber configuration to layer::udp as in other structs.

    * Added yaml creation out of a stringstream while building.

    * Made yaml-cpp optional again. Added compiler flags depending on ECAL_CORE_CONFIGURATION option to exclude yaml-cpp code.

commit 63d1cc2
Author: Rex Schilasky <[email protected]>
Date:   Tue Jul 30 16:04:12 2024 +0200

    [core] zero-copy-zero-payload-shm-transfer (eclipse-ecal#1683)

    * zero length payload was not transferred over shm in zero copy mode

commit 9b4c0d3
Author: KerstinKeller <[email protected]>
Date:   Tue Jul 30 14:53:07 2024 +0200

    [core] Protobuf Publisher Send should return the actual send size. (eclipse-ecal#1681)

commit 8f80696
Author: KerstinKeller <[email protected]>
Date:   Mon Jul 29 17:05:01 2024 +0200

    [build] ensure compatibility with yaml-cpp < 0.8.0 (eclipse-ecal#1678)

commit 0b60453
Author: Rex Schilasky <[email protected]>
Date:   Mon Jul 29 14:14:25 2024 +0200

    [core] registration-provider-refactoring (eclipse-ecal#1677)

    * registration provider single threaded sending*
    * Register(false) calls removed finally
    * registration provider logic reduced to just send registration samples (connection to descgate, monitoring cut)

commit 9110d2a
Author: Rex Schilasky <[email protected]>
Date:   Fri Jul 26 16:47:21 2024 +0200

    [core] registration-sender-datarace-fix (eclipse-ecal#1674)

commit 2c99f61
Author: Rex Schilasky <[email protected]>
Date:   Fri Jul 26 16:41:52 2024 +0200

    [core] serialization-decode-error-handling (eclipse-ecal#1673)

commit 0d361d0
Author: Peguen <[email protected]>
Date:   Fri Jul 26 14:27:00 2024 +0200

    [config] Introduction of YAML format for eCAL configuration  (eclipse-ecal#1669)

    * Changed runtime configuration format from ini to yaml
    * Implementation of yaml reader
    * Removing the simple-ini dependency + implementation from core
    * Implementation of defaults via yaml reading

commit 9b5f082
Author: Rex Schilasky <[email protected]>
Date:   Wed Jul 24 18:43:05 2024 +0200

    [core] active-registration-logic-getter (eclipse-ecal#1671)

commit 9211a88
Author: KerstinKeller <[email protected]>
Date:   Wed Jul 24 15:15:01 2024 +0200

    [core] further refactoring registration receiver. (eclipse-ecal#1670)

commit 21535da
Author: Rex Schilasky <[email protected]>
Date:   Mon Jul 22 13:39:01 2024 +0200

    [core] registration-loopback-logic-fixed-for-service-registrations (eclipse-ecal#1667)

    * registration sample processing (local/network mode, loopback not loopback) handle identically for pub/sub and client/server registration

    Co-authored-by: Kerstin Keller <[email protected]>

commit 3441e62
Author: Rex Schilasky <[email protected]>
Date:   Thu Jul 18 14:56:46 2024 +0200

    [core] new-pub-sub-matching-compatible (eclipse-ecal#1665)

commit 4659192
Author: KerstinKeller <[email protected]>
Date:   Wed Jul 17 18:01:20 2024 +0200

    [core] new-pub-sub-matching (eclipse-ecal#1653)

    Co-authored-by: Rex Schilasky <[email protected]>

commit 73f5f3c
Author: KerstinKeller <[email protected]>
Date:   Wed Jul 17 15:04:26 2024 +0200

    [core] refactor registration receiver and file structure (eclipse-ecal#1662)

commit 15d8354
Author: Rex Schilasky <[email protected]>
Date:   Wed Jul 17 14:47:05 2024 +0200

    [core] sync-with-ecal-core-2024-07-17 (eclipse-ecal#1664)

commit 5a835aa
Author: Peguen <[email protected]>
Date:   Mon Jul 15 16:10:32 2024 +0200

    [config] cleanup (eclipse-ecal#1655)

    * Removed unused methods from ecal_config.h
    * Removed unused member from transport_layer configuration.
    * Moved num_executor_reader/writer to publisher/subscriber config.
    * Added new num_execs to API and uses subscriber options for previous implementation calls.

commit 6dd1718
Author: Rex Schilasky <[email protected]>
Date:   Mon Jul 15 16:04:30 2024 +0200

    kill XProtect to avoid malware scan of dmg image (eclipse-ecal#1659)

commit a51ecec
Author: Rex Schilasky <[email protected]>
Date:   Mon Jul 15 11:48:23 2024 +0200

    [core] innosetup-configuration-path-fix (eclipse-ecal#1656)

    * innosetup path fix "configuration" -> "_configuration"
    * innosetup path fix "configinstall" -> "_configinstall"

commit 4d67a9f
Author: KerstinKeller <[email protected]>
Date:   Fri Jul 12 10:27:08 2024 +0200

    [core] registration provider refactoring (eclipse-ecal#1647)

    * Split logic for registration providers into different files.

commit a58f6af
Author: KerstinKeller <[email protected]>
Date:   Wed Jul 10 17:16:51 2024 +0200

    [core] Fix bug in frequency calculator that will reset the frequency to 0.0 even though data is still incoming. (eclipse-ecal#1650)

commit bd3bfc3
Author: Rex Schilasky <[email protected]>
Date:   Tue Jul 9 14:04:08 2024 +0200

    [core] shm-monitoring-performance (eclipse-ecal#1645)

    * shm_monitoring logic simplified, based on on switch only (needs to be improved further)
    * parallel shm_monitoring AND udp_monitoring feature removed
    * massive_pub_sub sample added to test creation performance of pub/sub for large scenarios and monitoring performance
    no more remvoval of expired values (erase_expired) in ApplyPub/Sub/Server/ClientDescription because of bad performance
    * erase_expired removal removed (we need to think about when/where to remove expired values)
    * shm registration configurable separately (interim solution to get this pr run)

commit 7b47978
Author: Rex Schilasky <[email protected]>
Date:   Fri Jun 28 11:46:39 2024 +0200

    [samples] shutdown condition fixed (eclipse-ecal#1641)

commit 35c1fd9
Author: Rex Schilasky <[email protected]>
Date:   Tue Jun 25 15:06:32 2024 +0200

    [core] new IsPublished API for CSubscriber, improved state logic for IsPublished and GetPublisherCount

commit 55eb50f
Author: KerstinKeller <[email protected]>
Date:   Tue Jun 25 15:04:08 2024 +0200

    [core] Refactoring / Renaming / Documentation of CExpMap  (eclipse-ecal#1637)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@rex-schilasky rex-schilasky rex-schilasky approved these changes

Assignees
No one assigned
Labels
cherry-pick-to-NONE Don't cherry-pick these changes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@KerstinKeller @rex-schilasky