How to activate 'ShapeFromSilhouette' in Default.xml?

[nkelsey86]

Ciao! I love this app so far, it is amazing.

For simple smooth objects (vase, coffe cup, etc) I think the Silhouette feature would work great, but I cannot get it to work.

I am trying to activate the 'ShapeFromSilhouette' feature by setting values in the Default.xml.

I do not see specifically how this can be set. I hope that it is possible to set this when sending to the 3DFlow processor.

Currently, I have been trying combinations of this:

<Category name="Stereo">
<Entry key="ShapeFromSilhouette" value="On"/> <-- should this be 'On' or 'true' or something else?
<Entry key="ShapeFromSilhouetteDiscretization?" value="medium"/> <-- should this be something else?
<Entry key="HyperplaneMatching?" value="On"/> <-- should this be 'On' or 'true' or something else?
</Category>?


I am including '_masked.jpg' versions of these files also (target in white, background in black).

Does this look right?

Thanks for the help,

- Nick

[Roberto]

Hi Nick,

The value "UseShapeFromSilhouette" and "UseShapeFromSilhouetteForInitialization" can be either set to "1" (true) or "0" (false).
I suggest setting UseShapeFromSilhouette -> 1 and UseShapeFromSilhouetteForInitialization -> 0.

The other value you can set are:

"ShapeFromSilhouetteLooseness" -> value from 0 to 100 (approximation factor, depends on your masks)
"ShapeFromSilhouetteNumberOfBins" -> values: 256, 512, 1024. 256 equals low, 512 medium, 1024 high

Please note that you don't need the hyperplane-matching (for standard stereo) as it has been dropped with the latest versions.

Best,
Roberto

[nkelsey86]

WOW - thanks so much for the quick response! I appreciate it!

[nkelsey86]

The job keeps failing on me though.

Here is my Default.xml file:
<?xml version="1.0" encoding="utf-8"?>
<Settings>
<Category name="Samantha">
<Entry key="Preset" value="General;Default"/>
</Category>
<Category name="Stereo">
<Entry key="UseShapeFromSilhouette" value="1"/>
<Entry key="UseShapeFromSilhouetteForInitialization" value="0"/>
<Entry key="ShapeFromSilhouetteLooseness" value="50"/>
<Entry key="ShapeFromSilhouetteNumberOfBins" value="256"/>
</Category>
<Category name="Surface">
<Entry key="Preset" value="General;Default"/>
</Category>
<Category name="Texture">
<Entry key="Preset" value="General;Default"/>
</Category>
</Settings>?


Here is the build error (Samantha phase runs okay):
1 2023-03-21T16:17:32.805Z FlowEngineAWSContainer failure. Quitting...
2 2023-03-21T16:17:32.321Z Generic error
3 2023-03-21T16:17:32.288Z [16:17:32] Input Sparse point cloud vertices: 73316
4 2023-03-21T16:17:32.288Z [16:17:32] Cameras: 300
5 2023-03-21T16:17:32.288Z [16:17:32] --- Settings ---
6 2023-03-21T16:17:32.288Z [16:17:32] Starting 3DF Stasia...
7 2023-03-21T16:17:32.288Z [16:17:32] Starting FlowEngine v. 6.010
8 2023-03-21T16:17:32.153Z Dense cloud generation...4 5
9 2023-03-21T16:17:32.121Z [16:17:32] 3DF Samantha ended successfully.?


Perhaps I have too many cameras? I have been sent 300 files to work with, is that too many?

Thanks,
Nick

[Roberto]

Since Flowengine failed at the beginning, maybe the camera masks were not loaded correctly. I have to ping Andrea Alessi as I'm not an expert of the aws version. On standard Flowengine, you load the camera masks with the CameraInterface::loadMask or CameraLoaderInterface::loadMasks functions. The number of cameras is not a problem. Also, I noted the version you are using is not up-to-date (6.010). I don't think it would solve this issue, but I also suggest using the latest one available.

Cheers,
Roberto

[nkelsey86]

I tried again, this time my .zip file only had 60 image/masks in it (with the same Default.xml settings as above)... I still get following error:

1 2023-03-21T17:43:41.903Z FlowEngineAWSContainer failure. Quitting...
2. 2023-03-21T17:43:41.777Z Generic error
3. 2023-03-21T17:43:41.772Z [17:43:41] Starting FlowEngine v. 6.010
4. 2023-03-21T17:43:41.772Z [17:43:41] Starting 3DF Stasia...
5. 2023-03-21T17:43:41.772Z [17:43:41] --- Settings ---
6. 2023-03-21T17:43:41.772Z [17:43:41] Cameras: 60
7. 2023-03-21T17:43:41.772Z [17:43:41] Input Sparse point cloud vertices: 19885?

[nkelsey86]

Sorry, I just saw your last comment (thanks again!).

My file & masks are loaded into the root folder of my .zip file, along with the Default.xml (contents provided above).

The image files are saved like the following: 0001.jpg, 0002.jpg, etc.
The corresponding masks are saved like this (in the same directory): 0001.jpg_mask.jpg, 002.jpg__mask.jpg, etc. (<-- perhaps this is the issue?)

[nkelsey86]

I tried this also with mask files renamed to '0001_mask.jpg, etc.' and the process still gives 'generic error' at very beginning of stacia process.

[Andrea Alessi]

have you tried with .bim images? I'll check, but IIRC FlowEngine cannot load non-bim masks.

[nkelsey86]

Ok, thanks. I thought in the documentation I found online you can load '_masked' versions of your target files, with black for background and white for target, and the system would use those.
It could be outdated or a different verison, but here is what I saw:

https://www.3dflow.net/zephyr-doc/en/3DFMasquerade.html

(bottom line): You may import masks that have been generated outside 3DF Masquerade into 3DF Zephyr using the _masked suffix.


Also in this thread it looks like the same advice, even for 3DFlow:

https://www.3dflow.net/forums/forum/3df-zephyr-forum-english/6366-bim-format-of-mask

*I'm probably missing something here, so sorry if it ends up being something obvious...

[Andrea Alessi]

3DF Zephyr can indeed load masks in that way, but not FlowEngine IIRC. They are two separate products although vastly compatible

[nkelsey86]

I tried using the 3D Flow example for silhouettes (the bottle example), just to see if I am getting things correct.

I created a .zip archive with the '.JPG' files and '.JPG.bim' files included in the sample you guys provide (e.g. 'IMG_0689.JPG' & 'IMG_0689.JPG.bim', etc.).

Also, I included a 'Default.xml' file with the following contents:

<?xml version="1.0" encoding="utf-8"?>
<Settings>

<Category name="Samantha">

<Entry key="Preset" value="General;Default"/>

</Category>
<Category name="Stereo">

<Entry key="UseShapeFromSilhouette" value="1"/>
<Entry key="UseShapeFromSilhouetteForInitialization" value="0"/>
<Entry key="ShapeFromSilhouetteLooseness" value="50"/>
<Entry key="ShapeFromSilhouetteNumberOfBins" value="256"/>

</Category>
<Category name="Surface">

<Entry key="Preset" value="General;Default"/>

</Category>
<Category name="Texture">

<Entry key="Preset" value="General;Default"/>

</Category>

</Settings>?

I checked in the S3 ingestion bucket and the folder & zip archive share same name, the files are same name as in the sample provided.
Could it be a problem with the 'Default.xml' file? Do I need more/less/different information?

Thanks and sorry to take up so much time.

P.S. - Is there an example 'Default.xml' that can be run for FlowEngine to render from silhouette on the bottle example?