Bug #328
closedMajor Appion database schema change
Added by Neil Voss over 14 years ago. Updated over 14 years ago.
100%
Description
We talked about this at our appion meeting. I want to change the refine tables to be more general for other processing packages.
Updated by Neil Voss over 14 years ago
- Changing Table Names:
ApRefinementData -> ApRefineIterData #fix confusing name ApRefinementRunData -> ApRefineRunData #shorten name ApParticleClassificationData -> ApRefineParticleData #fix confusing name ApRefinementParamsData -> ApEmanRefineIterData #EMAN only params ApStackParticlesData -> ApStackParticleData #rm plural ApAlignParticlesData -> ApAlignParticleData #rm plural ApClusteringParticlesData -> ApClusteringParticleData #rm plural
- Columns Moving to Different Table:
ApRefinementParamsData.symmetry -> ApRefineIterData.symmetry #these are global params not EMAN only ApRefinementParamsData.mask -> ApRefineIterData.mask #these are global params not EMAN only ApRefinementParamsData.imask -> ApRefineIterData.imask #these are global params not EMAN only
- Merging Columns:
ApRefinementData.classAverage ApRefinementData.emanClassAvg ApRefinementData.MsgPGoodClassAvg #barely used ApRefinementData.SpiCoranGoodClassAvg
There are 4 columns for class averages, I want to have a more generic 2 column system of:ApRefinementData.classAverage ApRefinementData.classAverage2
Or we can call it finalClassAverage, filteredClassAverage
ApParticleClassificationData.coran_keep -> ApRefineParticleData.filter_keep ApParticleClassificationData.msgp_keep -> ApRefineParticleData.filter_keep ApParticleClassificationData.thrown_out ~~ ApRefineParticleData.refine_keep #need to invert valueIt is good to keep track of which particle EMAN gets rid of "thrown_out" and which particles are rejected for other reason
"msgp_keep", "coran_keep", but they can be unified into "filter_keep", "refine_keep"
- Changing Column Names:
ApRefinementData.refinementParams -> ApRefineIterData.emanParams #eman only table ApRefinementData.refinementRun -> ApRefinementData.refineRun #shorten name ApRefinementRunData.name -> ApRefineRunData.runname #make consistent with other tables
- New Calculated Columns:
ApRefineRunData.num_iter #number of iterations ApStackData.boxsize #final boxsize of stack, makes it easier to get/change rather than calculate
- Project columns:
*.project|projects|project -> *.project
Can we do this?
Updated by Neil Voss over 14 years ago
- Target version set to Appion/Leginon 2.0.0
Updated by Anchi Cheng over 14 years ago
what is *.project? I don't see what that accomplishes since the table and column for it is not in sinedon format.
Merging columns
ApRefinementData.classAverage
ApRefinementData.emanClassAvg
ApRefinementData.MsgPGoodClassAvg #barely used
ApRefinementData.SpiCoranGoodClassAvg
I thought the last three were divided to make it easier to query individually. I don't remember the details.
If we are sure that package name is sufficient for knowing which ones were generated from what method, generic names are indeed simpler.
Updated by Neil Voss over 14 years ago
Actually the sinedon format would be:
REF|project|projects|projectId
NOT
project|projects|project
And it probably should be
REF|project|projects|DEF_id
to be really compatible with sinedon.
Currently the project Table is not compatible with Sinedon and all the fields linking to it are just an integers.
Updated by Neil Voss over 14 years ago
Comment on merging columns by merging we get some advantages:
- Programs that make substack new volumes based on class averages only need to link to one column not 3 or 4.
- The Recon Report page/particledata.inc will have less IF statements
- If type is of concern we can create a new field, classAverage2type, that will fix this
Updated by Anchi Cheng over 14 years ago
Actually sinedon format would be `REF|projectdata|projects|xxxx` xxxx refers to the key you used in the appiondata.py
where projectdata refers to the module name in sinedon.cfg not the database name.
About merging columns, I agree that it should work fine.
Updated by Neil Voss over 14 years ago
MyAMI Branch with Schema changes has been created.
svn co http://emg.nysbc.org/svn/myami/branches/may2010schema/ may2010schema/
Updated by Neil Voss over 14 years ago
- Status changed from Assigned to In Test
- Priority changed from High to Normal
- % Done changed from 70 to 100
Updated by Neil Voss over 14 years ago
- Assignee changed from Neil Voss to Anchi Cheng
test on Scott's data and see if it works.
Updated by Anchi Cheng over 14 years ago
- Status changed from In Test to Assigned
- Assignee changed from Anchi Cheng to Neil Voss
Scott's ApRefineGoodBadParticleData is empty after upgrade.
Updated by Neil Voss over 14 years ago
Added schema-14077.py to do this. It can be done any time, but the code is already ready written.
source:trunk/dbschema/schema-14077.py
Updated by Neil Voss over 14 years ago
- Status changed from Assigned to In Code Review
- Assignee changed from Neil Voss to Anchi Cheng
Added schema-14077.py to do this. It can be done any time, but the code is already ready written for the latest version.
source:trunk/dbschema/schema-14077.py
Updated by Anchi Cheng over 14 years ago
- Status changed from In Code Review to In Test
added version and dbcheck in r14096. Tested at the same time as testing bug #479
Updated by Anchi Cheng over 14 years ago
Why can't I update this issue to closed?
Updated by Anchi Cheng over 14 years ago
- Assignee changed from Anchi Cheng to Amber Herold
I can't close this issue even though it is tested.
Updated by Amber Herold over 14 years ago
You must close #345 before you can close this one because it is blocking this.
Updated by Amber Herold over 14 years ago
- Status changed from In Test to Closed