Skip to content

Add data product to record filtering fraction#1691

Open
brownd1978 wants to merge 7 commits intoMu2e:mainfrom
brownd1978:filtfrac
Open

Add data product to record filtering fraction#1691
brownd1978 wants to merge 7 commits intoMu2e:mainfrom
brownd1978:filtfrac

Conversation

@brownd1978
Copy link
Collaborator

This is intended to record prescale factors, but is more generalized in case there are other uses.

@FNALbuild
Copy link
Collaborator

Hi @brownd1978,
You have proposed changes to files in these packages:

  • DataProducts
  • Filters

which require these tests: build.

@Mu2e/write, @Mu2e/fnalbuild-users have access to CI actions on main.

⌛ The following tests have been triggered for 33c5055: build (Build queue - API unavailable)

About FNALbuild. Code review on Mu2e/Offline.

@FNALbuild
Copy link
Collaborator

☀️ The build tests passed at 33c5055.

Test Result Details
test with Command did not list any other PRs to include
merge Merged 33c5055 at 440d6ec
build (prof) Log file. Build time: 09 min 30 sec
ceSimReco Log file.
g4test_03MT Log file.
transportOnly Log file.
POT Log file.
g4study Log file.
cosmicSimReco Log file.
cosmicOffSpill Log file.
ceSteps Log file.
ceDigi Log file.
muDauSteps Log file.
ceMix Log file.
rootOverlaps Log file.
g4surfaceCheck Log file.
trigger Log file.
check_cmake Log file.
FIXME, TODO TODO (0) FIXME (0) in 2 files
clang-tidy ➡️ 2 errors 4 warnings
whitespace check no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 33c5055 after being merged into the base branch at 440d6ec.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

Copy link
Contributor

@sophiemiddleton sophiemiddleton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Dave, this definitely looks useful!

Copy link
Contributor

@michaelmackenzie michaelmackenzie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good and is very useful, thank you for making this update!

Can you please add this data product output to the Trigger/src/PrescaleEvent_module.cc prescale module? This may require a separate object for On-Spill and Off-Spill in this case.

@michaelmackenzie
Copy link
Contributor

I can also make the update to the Trigger prescaler module after this PR if that's easier!

@brownd1978
Copy link
Collaborator Author

Can you please add this data product output to the Trigger/src/PrescaleEvent_module.cc prescale module? This may require a separate object for On-Spill and Off-Spill in this case.

Can the OnSpill/OffSpill split be handled by having 2 instances of this module in different paths with the appropriate filter before them?

@michaelmackenzie
Copy link
Contributor

Can you please add this data product output to the Trigger/src/PrescaleEvent_module.cc prescale module? This may require a separate object for On-Spill and Off-Spill in this case.

Can the OnSpill/OffSpill split be handled by having 2 instances of this module in different paths with the appropriate filter before them?

I'd prefer for now to keep the trigger configuration as it is, without needing to duplicate all paths (or more multiplying if there are any other event modes) in order to have different pre-scales On-Spill and Off-Spill. The Trigger prescaler is designed to be able to handle different pre-scales for different event modes, without duplication.

@brownd1978
Copy link
Collaborator Author

I think I addressed all the issues. I tested with on OnSpill digitization job (with Production PR 498) and see the new products look as expected.

@brownd1978
Copy link
Collaborator Author

Example filedumper output:

PRINCIPAL TYPE: SubRun
PROCESS NAME | MODULE LABEL............ | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE......... | PRODUCT FRIENDLY TYPE..... | SIZE
Primary..... | genCounter.............. | ..................... | mu2e::GenEventCount....... | mu2e::GenEventCount....... | ...-
Digitize.... | MprTrkDe80m70pD0200PS... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe50D0200PS....... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe70D0200PS....... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTwoTrkDe50PS......... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprHlxDe30pIPAPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | MinBiasSDCountPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe75m70pD0200PS... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprTrkDe80m70pD0200PS... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe80m70pD0200PS... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe75D0200PS....... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTwoTrkDe50PS......... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprHlxDe50PS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprHlxDe30pIPAPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe75m70pPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprTrkDe80m70pD0200PS... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloPhotonPS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe75D0200PS....... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloMVANNCEPS........... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe80m70pD0200PS... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe70PS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CstTimeClusterPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe50D0200PS....... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTwoTrkDe80m70pD0200PS | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe75m70pPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprHlxDe30pIPAPhiPS..... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe75m70pD0200PS... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloN0SourcePS.......... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkUe80m70pPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprHlx30PS.............. | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprHlxDe50PS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | MinBiasSDCountPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprTrkDe50D0200PS....... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe70PS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | MinBiasCDCountPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe75m70pPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | LumiStreamPS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | MinBiasCDCountPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprTrkDe80m70pPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloN0SourcePS.......... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprHlxDe70m50pPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe75D0200PS....... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | MprTrkDe80m70pD0200PS... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprTrkDe50D0200PS....... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprHlxUe40PS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe50D0200PS....... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe75m70pPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprHlxDe70m50pPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CstCosmicTrackSeedPS.... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | PBISim.................. | MeanIntensity........ | mu2e::ProtonBunchIntensity | mu2e::ProtonBunchIntensity | ...-
Digitize.... | CprTrkDe80m70pPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloRMCPS............... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe70PS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe80m70pD0200PS... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprHlx30PS.............. | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe70PS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe75m70pD0200PS... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTwoTrkDe80m70pD0200PS | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprHlxUe50m30pPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloCosmicPS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprHlxUe40PS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloPhotonPS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe80m70pPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe70D0200PS....... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprHlx50PS.............. | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe80m70pD0200PS... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprTrkDe80m70pPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprHlx50Hlx30PS......... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloRMCPS............... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloCosmicPS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe75PS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe75PS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe75PS............ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprHlxDe30pIPAPhiPS..... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe75m70pD0200PS... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CstCosmicTrackSeedPS.... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkDe80m70pPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprTrkUe80m70pPS........ | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprHlx50Hlx30PS......... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe75PS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe80m70pPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe70D0200PS....... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | AprHlx50PS.............. | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | TprHlxUe50m30pPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe50D0200PS....... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CaloMVANNCEPS........... | OnSpill.............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe70D0200PS....... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | LumiStreamPS............ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CstTimeClusterPS........ | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-
Digitize.... | CprTrkDe75D0200PS....... | OffSpill............. | mu2e::FilterFraction...... | mu2e::FilterFraction...... | ...-

Copy link
Contributor

@michaelmackenzie michaelmackenzie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good and I appreciate you adding the ability to add the filter fraction options. I only requested minor edits to preserve the 100% prescale mode used in Online processing

++_npass;
++nevt_[imode];
// Apply the prescale
bool retval = e.event() % mode.prescale_ == 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use ps <= 0 as a special case to entirely prescale a path while still loading the modules and having the path in the art::TriggerResults. Therefore, we need to check this before apply the modulo.

Copy link
Collaborator Author

@brownd1978 brownd1978 Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused about how negative prescales are intended to work. AFAIK
bool retval = e.event() % mode.prescale_ == 0;
will produce the same result for negative or positive prescale values of the same magnitude.
What do you want the code to do?

I honestly dont see a use case here for negative values, espectially since 0 does what you said above, and users might be confused by negative prescales. Making this an unsigned seems cleaner.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current functionality is prescale values <= 0 indicate deactivating the path. This is used to disable paths for specific event modes (e.g. On-Spill) while allowing for processing in alternate event modes (e.g. Off-Spill). The current menus all default path prescales to -1 and so we need to protect against this. Also, a prescale of 0 would cause issues here as well as <value> % 0 is undefined:

root [0] 6 % 0
ROOT_prompt_0:1:3: warning: remainder by zero is undefined [-Wdivision-by-zero]
6 % 0
  ^ ~
(int) 8

bool PrescaleEvent::endSubRun( art::SubRun& subrun ) {
for (size_t imode = 0; imode < eventMode_.size(); imode++){
auto const& mode = eventMode_[imode];
double frac = 1.0/double(mode.prescale_);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This prescale value could be <= 0, so this should be checked here as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added protection against prescale == 0, I agree that's a legitimate use case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I see where this protection is, above it I still see:

 double frac = 1.0/double(mode.prescale_);

but it should probably be:

const double frac = (mode.prescale_ > 0) ? 1.0/double(mode.prescale_) : 0.;

@brownd1978
Copy link
Collaborator Author

@FNALbuild run build test

@FNALbuild
Copy link
Collaborator

⌛ The following tests have been triggered for 680c5d0: build (Build queue - API unavailable)

@FNALbuild
Copy link
Collaborator

☔ The build tests failed for 680c5d0.

Test Result Details
test with Command did not list any other PRs to include
merge Merged 680c5d0 at 4f1ca06
build (prof) Log file. Build time: 04 min 34 sec
ceSimReco Log file.
g4test_03MT Log file.
transportOnly Log file.
POT Log file.
g4study Log file.
cosmicSimReco Log file.
cosmicOffSpill Log file.
ceSteps Log file.
ceDigi Log file.
muDauSteps Log file.
ceMix Log file.
rootOverlaps Log file.
g4surfaceCheck Log file.
trigger Log file. Return Code 2.
check_cmake Log file. Return Code 1.
FIXME, TODO TODO (0) FIXME (0) in 7 files
clang-tidy ➡️ 7 errors 19 warnings
whitespace check no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 680c5d0 after being merged into the base branch at 4f1ca06.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

@brownd1978
Copy link
Collaborator Author

@FNALbuild run build test

@FNALbuild
Copy link
Collaborator

⌛ The following tests have been triggered for 680c5d0: build (Build queue - API unavailable)

@FNALbuild
Copy link
Collaborator

☔ The build tests failed for 680c5d0.

Test Result Details
test with Command did not list any other PRs to include
merge Merged 680c5d0 at 4f1ca06
build (prof) Log file. Build time: 04 min 48 sec
ceSimReco Log file.
g4test_03MT Log file.
transportOnly Log file.
POT Log file.
g4study Log file.
cosmicSimReco Log file.
cosmicOffSpill Log file.
ceSteps Log file.
ceDigi Log file.
muDauSteps Log file.
ceMix Log file.
rootOverlaps Log file.
g4surfaceCheck Log file.
trigger Log file.
check_cmake Log file. Return Code 1.
FIXME, TODO TODO (0) FIXME (0) in 7 files
clang-tidy ➡️ 7 errors 19 warnings
whitespace check no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 680c5d0 after being merged into the base branch at 4f1ca06.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

Copy link
Contributor

@michaelmackenzie michaelmackenzie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for making these updates!

@brownd1978
Copy link
Collaborator Author

@FNALbuild run build test

@FNALbuild
Copy link
Collaborator

⌛ The following tests have been triggered for 3476511: build (Build queue - API unavailable)

@FNALbuild
Copy link
Collaborator

☔ The build tests failed for 3476511.

Test Result Details
test with Command did not list any other PRs to include
merge Merged 3476511 at 45ae2d8
build (prof) Log file. Build time: 11 min 56 sec
ceSimReco Log file.
g4test_03MT Log file.
transportOnly Log file.
POT Log file.
g4study Log file.
cosmicSimReco Log file.
cosmicOffSpill Log file.
ceSteps Log file.
ceDigi Log file.
muDauSteps Log file.
ceMix Log file.
rootOverlaps Log file.
g4surfaceCheck Log file.
trigger Log file.
check_cmake Log file. Return Code 1.
FIXME, TODO TODO (0) FIXME (0) in 7 files
clang-tidy ➡️ 7 errors 19 warnings
whitespace check no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 3476511 after being merged into the base branch at 45ae2d8.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

@brownd1978
Copy link
Collaborator Author

@FNALbuild run build test

@FNALbuild
Copy link
Collaborator

⌛ The following tests have been triggered for a14efe1: build (Build queue - API unavailable)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants