Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sample feature #48

Closed
3 tasks done
Tracked by #40
plastikfan opened this issue Oct 25, 2023 · 6 comments
Closed
3 tasks done
Tracked by #40

sample feature #48

plastikfan opened this issue Oct 25, 2023 · 6 comments
Assignees
Labels
feature New feature or request

Comments

@plastikfan
Copy link
Contributor

plastikfan commented Oct 25, 2023

Allows user to try out multiple profiles in the same batch to see which one works the best. Requires a sample entry in the config which is just an array of string. Each string referrers to a profile

The sample config also needs a number of images to try on (let's default to 3) and a minimum image size in MB (default=1).

For each of the images run the selected profile.

By default, the default directory is the current one. For each file we create a relative path that contains the profile name. The out file name by default s the same as the input file, but this can be changed by a config setting

Actually we can create many more config settings without necessarily creating flags. This will avoid complicating the cli.


Some more design notes:

  • let's delete the Mode param as it is too vague
  • we introduce 2 concepts, the OutputStrategy and DeletionStrategy, each of which will implement an Inline and Mirror strategy. (as we'll see later, they can be mixed with each other)
  • The runner is aware of the output strategy moving files accordingly, using the path-finder to create the paths and the file-manager to interact with the file system, using a vfs.
  • OutputStrategy(INLINE): the output file(s) are created in the same location as item.Path (but do we create as the original name or as an alternative output name; this would also require renaming the original as required)
  • OutputStrategy(MIRROR): the output files(s) are created in mirror path,, but we also re-create the relative directory of the source to implement mirroring
  • DeletionStrategy(INLINE): the Trash folder is in the same directory as item.Path
  • DeletionStrategy(MIRROR): the Trash is in the a mirrored relative directory under MirrorPath
  • Since, the OutputStrategy and DeletionStrategy are independent of each other, they can be mixed and matched as required; ie just because you're using Mirror in one context, doesn't mean you have to use Mirror in the other.
  • perhaps we have a transparency mode, ie perform renames such that the new generated files adopt the existing files, so there is no difference, except for the original file would be renamed to something else. With transparency enabled, we make all the decisions to make this possible; we internally make the choice of which strategies are in place, so the user doesn't have to work this out for themselves. But the deletion strategy is independent of transparency, so it really only applies to output. Or, perhaps, do we assume transparent by default and the other options adjust this.
  • In scenarios where we need to rename the original file to something else, by default, we append "LEGACY" to the end. This can be configured in advanced settings
  • Advanced settings, a new section in config contains settings for which there are no command line flags, as they rarely need to be changed
  • the strategies are properties of the path finder as they help create paths depending on usage mode

Further tasks:

@plastikfan plastikfan self-assigned this Oct 25, 2023
@plastikfan plastikfan added the feature New feature or request label Oct 25, 2023
@plastikfan plastikfan changed the title try feature sample feature Oct 25, 2023
@plastikfan
Copy link
Contributor Author

plastikfan commented Dec 2, 2023

found this interesting article about workflow patterns: Control-Flow Patterns.

The reason why I'm considering this, is because I'm looking at the classes required to implement the sampling feature. So far, I have come up with the followng abstractions (the names are not necessarily the ones that will be used):

  • Manager/Controller : on the occurence of a new TraverserItem (deleivered by ShrinkEntry), controls how it is handled. IE, it knows the sequence of events that should subsequently occur. For sampling, this is more complicated than a full run, becuase for each new TraverseItem, multiple invocations of the third party command (magick) are required. For a full run, we only require a single external invoke.So it will drive the 'workfow' and in this case means, rename existing file, collating the correct third party args and issuing the command and waiting for the result. (There will be 2 different instances of the controller, 1 for sampling and the other for a full run; these will be hidden behind an interface)
  • FileManager : responsible for working with the file system, so will contains the virtual file system reference
  • PathFinder : concerned with creating correct paths according to incoming paramters

Actually, the referenced site looks to be very good as it deals with scenarios that I have been wrestling with in various parts of my snivilised projects, but I have dealt with them on an adhoc manner, coming up with my own way of doing things rather that looking at established patterns. This can serve as a very good reference. A lot of these pattarns are relevant for web workfows, but they can be adapted to be used in other contexts too.

@plastikfan
Copy link
Contributor Author

some temporary code:

func (pc ProfilesConfig) AsCommandLine(name string) cobrass.ThirdPartyCommandLine {
	const approx = 2

	if name != "" {
		if profile, found := pc[name]; !found {
			keys := profile.Keys()
			size := len(keys)
			cl := make(cobrass.ThirdPartyCommandLine, 0, size*approx)

			for flag, optionValue := range keys {
				long := fmt.Sprintf("--%v", flag)
				cl = append(cl, long)

				if !slices.Contains(booleanValues, optionValue) {
					cl = append(cl, optionValue)
				}
			}

			return cl
		}
	}

	return cobrass.ThirdPartyCommandLine{}
}

func (pc ProfilesConfig) WithDashes(name string) clif.SpecifiedFlagsCollection {
	if name != "" {
		if profile, found := pc[name]; found {
			flags := clif.SpecifiedFlagsCollection{}

			for _, flag := range profile.Keys() {
				optionValue := profile[flag]
				long := fmt.Sprintf("--%v", flag)
				flags[long] = optionValue
			}

			return flags
		}
	}

	return clif.SpecifiedFlagsCollection{}
}

@plastikfan
Copy link
Contributor Author

plastikfan commented Dec 14, 2023

NB: we can open a folder (on mac), (for pixa, typically the output folder) from the command line using:

open <path-name>

On windows (yet to be confirmed):

start <path-name>

Not sure what the linux equivalent is yet...

@plastikfan
Copy link
Contributor Author

plastikfan commented Dec 14, 2023

Need to start using mocking, in this case so we can mock out viper config. Using uber mock.

The first to know, is how to make a mock; here is the documented example (as described in mock_test.go):

mockgen -destination mock_test.go -package gomock_test -source example_test.go

  • use -mock_names to specify the interface to be mocked and the mock name, eg:

Repository=MockSensorRepository,Endpoint=MockSensorEndpoint, where Repository is the interface name and MockSensorRepository is the desired mock name

  • --source specifies where the source interface is defined, eg:
    -source=foo.go

But actually, this doesnt clearly explain how to mock a foreign interface. I found this: codecentric.de gomock-tutorial, which may be a bit clearer, in particular:

mockgen -destination=mocks/mock_doer.go -package=mocks github.com/sgreben/testing-with-gomock/doer Doer

where:

  • mocks: is the name of a package that will contain our mocks (mocks are to be created in a specific mock package)
  • there are 2 non flag arguments, an import path, and a comma-separated list of symbols
  • in the above example, github.com/sgreben/testing-with-gomock/doer is the import path and Doer is the symbol to be mocked

To make this a bit easier, we should not try to create a mock for a foreign interface as this is complicating the task. Create the mock in CObrass and then export it so it can be resued by clients.

@plastikfan
Copy link
Contributor Author

Actually, after having implemented a mock for viper config, I realised its better to re-organise the profiles and sampler config. They should be wrapped behind interfaces then mocks should be generated for those. The problem with mocking viper config is that its difficult to re-implement the marshalling when really, we need to by-pass marshalling altogether.

@plastikfan
Copy link
Contributor Author

this feature looks to be too big to do under a single issue; therefore it will be split up. This issue will deal just with the default scenario, which is transparent mode which means:

  • inline mode for output generation and input deletion
  • input file moved to local TRASH folder
  • output generated in local location including mirror-sub-path and supplement(profile/scheme)

plastikfan added a commit that referenced this issue Dec 20, 2023
plastikfan added a commit that referenced this issue Dec 20, 2023
feat(test): move test folder and provide other test setup (#48)

feat(proxy): add registry, runner and other supporting functionality (#48)

feat(proxy): setup scientist directory for testing (#48)

feat(proxy): move program execution to runner (#48)

feat(proxy): add profile/scheme validation (#48)

feat(proxy): clean up root param set/root inputs (#48)

feat(proxy): validate sampler info (#48)

feat(proxy): change profile params to be a map for cobrass.ThirdPartyCommandLine (#48)

feat(proxy): add initial sequencing functionality (#48)

chore(deps): incoming breaking changes from extendio & cobrass (#48)

ref(proxy): reorganise runner (#48)

test(proxy): migrate tests to use virtual-fs (#48)

test(proxy): mock viper config (#48)

test(proxy): mock configs (#48)

ref(proxy): redesign configs so they can be mocked (#48)

test(proxy): mock config readers (#48)

ref(proxy): add supporting code for strategies and some tidy up (#48)

ref(proxy): rename mirror-path to output (#48)

feat(proxy): add trash flag (#48)

feat(proxy): add vfs and implement strategies (#48)

feat(proxy): add basic path-finder transparent mode (#48)

feat(proxy): tidy up (#48)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant