V0.0.1 #13

ctr26 · 2023-08-30T14:20:17Z

Still not fixed pytest errors but this is better than it was

…aybe make this a seperate module/repo

…resources

ctr26 · 2023-09-02T11:47:07Z

@afoix Fixed deps I think

ctr26 · 2023-09-06T10:30:55Z

import torch.multiprocessing
torch.multiprocessing.set_sharing_strategy('file_system')

@afoix

ctr26 · 2023-09-06T10:43:08Z

if name=="__main__":

Fix this

ctr26 · 2023-09-06T10:43:15Z

Pillow=9.5.0

afoix · 2023-09-15T16:40:03Z

@ctr26 the changes suggested above are pushed in that branch now - they were working in the beast

ctr26 · 2023-09-15T16:40:37Z

import torch.multiprocessing
torch.multiprocessing.set_sharing_strategy('file_system')

Does this make a difference?

afoix · 2023-09-18T15:32:32Z

scripts/train.py

@@ -97,9 +97,9 @@
 # dataloader = DataLoader(train_dataset, batch_size=batch_size,
 #                         shuffle=True, num_workers=2**4, pin_memory=True, collate_fn=my_collate)

-model = Bio_VAE("VQ_VAE", channels=1, num_residual_layers=8, num_residual_hiddens=64)
+model = BioimageEmbed("VQ_VAE", channels=1, num_residual_layers=8, num_residual_hiddens=64)


here in in the parameter "VQ_VAE" would be possible to use the model we want? If yes, where are the other models set (or the list of possible models)?

All the models I allow are stored in:

https://github.com/ctr26/bioimage_embed/blob/v0.0.1/bioimage_embed/models/factory.py

"VQ_VAE" is deprecated now I think

afoix · 2023-09-18T15:33:09Z

scripts/train.py


-model = Bio_VAE("VQ_VAE", channels=1)
+model = BioimageEmbed("VQ_VAE", channels=1)


Why do you overwrite model soo soon?

afoix · 2023-09-18T15:36:35Z

scripts/train.py

Overall, I like more the new of the classes and I think those are the only changes so far in that file. So it looks good to me 👍🏻

afoix · 2023-09-18T15:37:44Z

scripts/shapes/shape_embed.py

+
+# Deal with the filesystem
+import torch.multiprocessing
+torch.multiprocessing.set_sharing_strategy('file_system')


This still pending of confirmation if actually makes a different when tested in another computer than the beast.

afoix · 2023-09-18T15:39:08Z

scripts/shapes/shape_embed.py

+        # "window_size": 64*2,
+        "num_workers": 1,
+        "input_dim": (1, window_size, window_size),
+        # "channels": 3,


Why do you comment channels isn't it a needed parameter?

Channels is (now) handled by "input_dim", channels used to be it's own param in my models because thats been changed now

afoix · 2023-09-18T15:48:10Z

scripts/shapes/shape_embed.py

+    for idx in range(len(dataset)):
+        try:
+            image, label = dataset[idx]
+            # If the transform works without errors, add the index to the list of valid indices
+            valid_indices.append(idx)
+        except Exception as e:
+            print(f"Error occurred for image {idx}: {e}")


I like a lot the try/except that you use here, good practise 🎉

afoix · 2023-09-18T15:48:45Z

scripts/shapes/shape_embed.py

+        batch_size=args.batch_size,
+        shuffle=True,
+        num_workers=args.num_workers,
+        # transform=transform,


why don't you use transform anymore here? Is it because you are using it previously?

The dataset has already been transformed from Images to distance_matrix tensors.
Ordinarily I would put the transform in the dataloader but here I use the transform to check for valid images too as conversion process doesnt always work ln:231

afoix · 2023-09-18T15:48:59Z

scripts/shapes/shape_embed.py

+
+    # dataloader = DataLoader(train_dataset, batch_size=batch_size,
+    #                         shuffle=True, num_workers=2**4, pin_memory=True, collate_fn=my_collate)


Suggested change

# dataloader = DataLoader(train_dataset, batch_size=batch_size,

# shuffle=True, num_workers=2**4, pin_memory=True, collate_fn=my_collate)

afoix · 2023-09-18T15:49:37Z

scripts/shapes/shape_embed.py

+    # model = Mask_VAE("VAE", 1, 64,
+    #                      #  hidden_dims=[32, 64],
+    #                      image_dims=(interp_size, interp_size))
+
+    # model = Mask_VAE(VAE())
+    # %%
+    # lit_model = LitAutoEncoderTorch(model)


Suggested change

# model = Mask_VAE("VAE", 1, 64,

# # hidden_dims=[32, 64],

# image_dims=(interp_size, interp_size))

# model = Mask_VAE(VAE())

# %%

# lit_model = LitAutoEncoderTorch(model)

afoix · 2023-09-18T15:52:05Z

scripts/shapes/shape_embed.py

+    # %%
+
+
+    # model_name = model._get_name()


Suggested change

# model_name = model._get_name()

ctr26 · 2023-09-20T07:40:01Z

@ctr26 remove all deprecations and commented lines

ctr26 · 2023-09-25T12:23:21Z

@afoix merged

ctr26 added 11 commits August 26, 2023 19:26

renaming to bioimage_embed

77f85d9

renaming modules to biovae

d1b0c4c

renaming class to BioImageEmbed

de5f1dc

updating readme

4f6461d

fixing naming and updating readme

0b39143

Making the readme more hype

1d22807

typo in pyproject

828f65c

fixing install

9b384fe

python 3.9

aa55682

removing poetry lock cache

75ab575

removing poetry lock cache and file

c9d3389

afoix self-requested a review September 1, 2023 09:18

afoix added the good first issue Good for newcomers label Sep 1, 2023

ctr26 added 9 commits September 1, 2023 15:08

Likely only works on python 3.9-3.10

f685626

simplifying testing

aa8c60f

using conda instead to manage poetry and python vers

7c16562

removing tests for this release, 0.0.2 will have passing tests

2adceac

Moving shapemebed to shapes scripts, means we can readme better and m…

12955b3

…aybe make this a seperate module/repo

conda needs bash -l

41c80a1

python = 3.9 for ease and for now

c412522

Use mamba, though the issue looks like github actions running out of …

b5b4034

…resources

I think this fixes the version mismatch

c8137b4

ctr26 marked this pull request as draft September 2, 2023 11:49

ctr26 marked this pull request as ready for review September 2, 2023 11:49

afoix mentioned this pull request Sep 4, 2023

Add some information about the kaggle API #14

Closed

Labelled broad data using snakemake

2f1af7b

add quick Craig's suggestions

3090e05

afoix approved these changes Sep 19, 2023

View reviewed changes

ctr26 and others added 8 commits September 20, 2023 10:53

Updating ignore

dfdc48c

Cleaning shapes script

0c2a71f

Cleaning inference ivy gap

f8407e0

Cleaning and move other ivy gap

7a6fa42

Removing old trash

3d2a9fb

Cleaning pythae example

b94ab58

Adding some legacy tests that don't work currently

164b7e1

I think this is the correct gitignore

df83d9e

ctr26 merged commit 73c145d into master Sep 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

V0.0.1 #13

V0.0.1 #13

ctr26 commented Aug 30, 2023

ctr26 commented Sep 2, 2023

ctr26 commented Sep 6, 2023 •

edited

Loading

ctr26 commented Sep 6, 2023

ctr26 commented Sep 6, 2023

afoix commented Sep 15, 2023

ctr26 commented Sep 15, 2023

afoix Sep 18, 2023

ctr26 Sep 20, 2023

afoix Sep 18, 2023

afoix Sep 18, 2023

afoix Sep 18, 2023

afoix Sep 18, 2023

ctr26 Sep 20, 2023

afoix Sep 18, 2023

afoix Sep 18, 2023

ctr26 Sep 20, 2023

afoix Sep 18, 2023

afoix Sep 18, 2023

afoix Sep 18, 2023

ctr26 commented Sep 20, 2023

ctr26 commented Sep 25, 2023


		model = Bio_VAE("VQ_VAE", channels=1)
		model = BioimageEmbed("VQ_VAE", channels=1)


		# dataloader = DataLoader(train_dataset, batch_size=batch_size,
		# shuffle=True, num_workers=2**4, pin_memory=True, collate_fn=my_collate)

V0.0.1 #13

V0.0.1 #13

Conversation

ctr26 commented Aug 30, 2023

ctr26 commented Sep 2, 2023

ctr26 commented Sep 6, 2023 • edited Loading

ctr26 commented Sep 6, 2023

ctr26 commented Sep 6, 2023

afoix commented Sep 15, 2023

ctr26 commented Sep 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ctr26 commented Sep 20, 2023

ctr26 commented Sep 25, 2023

ctr26 commented Sep 6, 2023 •

edited

Loading