Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
batch_uploading_oais_to_oclc_and_aleph [2020/04/29 23:05] ldegozzaldi |
batch_uploading_oais_to_oclc_and_aleph [2020/05/01 22:15] ldegozzaldi |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Batch Uploading OAIs from Scholarworks into OCLC and Aleph ====== | + | ===== Batch Uploading OAIs from Scholarworks into OCLC and Aleph ===== |
| ==== CHANGE TITLE? ETDs (Current) Processing ScholarWorks OAIs ==== | ==== CHANGE TITLE? ETDs (Current) Processing ScholarWorks OAIs ==== | ||
| NOTE: My helpful "hints" will appear in //Italics.// | NOTE: My helpful "hints" will appear in //Italics.// | ||
| - | **Introduction** | + | ===Introduction=== |
| The Graduate School will email "Packing Lists" dated February, May and September (end of semesters) of new dissertations, theses, MFA theses and occasionally LARP theses. There may be a lag between these dates and when the ETDs are available on ScholarWorks. //I try to process them after a couple of months have passed, to assure that they will be picked up in the Crosswalk harvest.// | The Graduate School will email "Packing Lists" dated February, May and September (end of semesters) of new dissertations, theses, MFA theses and occasionally LARP theses. There may be a lag between these dates and when the ETDs are available on ScholarWorks. //I try to process them after a couple of months have passed, to assure that they will be picked up in the Crosswalk harvest.// | ||
| - | **Preparation** | + | ===Preparation=== |
| * Have a handy copy (either online or a printout) of the Packing List-in-process. **NOTE:** It's a good idea to save copies of these in appropriate folders. Example: **PackingListReport_Feb2019diss.xlsx** in [Drive]:\OAI\Dissertations\2019\ (i.e. 2019), or OAI\Theses, ThesesMFA or ThesesLARP. | * Have a handy copy (either online or a printout) of the Packing List-in-process. **NOTE:** It's a good idea to save copies of these in appropriate folders. Example: **PackingListReport_Feb2019diss.xlsx** in [Drive]:\OAI\Dissertations\2019\ (i.e. 2019), or OAI\Theses, ThesesMFA or ThesesLARP. | ||
| * Open MarcEdit. (**NOTE:** Make sure your MarcEdit XSLT engine is set to SAXON.NET. On MarcEdit home page, click tools(found on top), Preferences, MARCEngine, select SAXON/NET under XSLT Engine.) | * Open MarcEdit. (**NOTE:** Make sure your MarcEdit XSLT engine is set to SAXON.NET. On MarcEdit home page, click tools(found on top), Preferences, MARCEngine, select SAXON/NET under XSLT Engine.) | ||
| - | **Harvesting from ScholarWorks** | + | ===Harvesting from ScholarWorks=== |
| - **Click on Harvest OAI Records:** (Found on either the MarcEdit home page or under (top) tools/OAI Harvester Tools/) Set the following: | - **Click on Harvest OAI Records:** (Found on either the MarcEdit home page or under (top) tools/OAI Harvester Tools/) Set the following: | ||
| * Server address: https://scholarworks.umass.edu/do/oai/ | * Server address: https://scholarworks.umass.edu/do/oai/ | ||
| Line 27: | Line 27: | ||
| * **IMPORTANT NEW STEP, added 2020:**Go to ScholarWorks/Dissertations and Theses and log onto "My account", scroll down to the appropriate series (i.e., DOCTORAL DISSERTATIONS (dissertations_2)/Manage Dissertations/Batch revise Excel/Generate a spreadsheet of current data. See [[changing_one_year_campus_access_titles_to_open_access|Changing one year campus titles to open access in ScholarWorks]] for instructions on generating ScholarWorks spreadsheets. If extra names appear in the MarcEdit file, check the generated spreadsheet to make sure they are NOT dated in the range requested. //(This step has been added since occasionally a dissertation or thesis will have been left off the Packing List.)// | * **IMPORTANT NEW STEP, added 2020:**Go to ScholarWorks/Dissertations and Theses and log onto "My account", scroll down to the appropriate series (i.e., DOCTORAL DISSERTATIONS (dissertations_2)/Manage Dissertations/Batch revise Excel/Generate a spreadsheet of current data. See [[changing_one_year_campus_access_titles_to_open_access|Changing one year campus titles to open access in ScholarWorks]] for instructions on generating ScholarWorks spreadsheets. If extra names appear in the MarcEdit file, check the generated spreadsheet to make sure they are NOT dated in the range requested. //(This step has been added since occasionally a dissertation or thesis will have been left off the Packing List.)// | ||
| * Any harvested record NOT on the Packing List that is also not on the generated spreadsheet, or has a different date (Check **degree_year** and **award_month**), or which belongs to a different series (such as English MFAs)can be removed from the MarcEdit file. | * Any harvested record NOT on the Packing List that is also not on the generated spreadsheet, or has a different date (Check **degree_year** and **award_month**), or which belongs to a different series (such as English MFAs)can be removed from the MarcEdit file. | ||
| - | **Edit the MarcEdit .mrk file of harvested records** | + | ===Edit the MarcEdit file of harvested records=== |
| - **Run MarcEdit task** | - **Run MarcEdit task** | ||
| * Change date in 008 with the new year, under Tools -> Manage Tasks -> Selected desired task in Task Lists window -> Manage Existing Tasks -> Edit Selected Task List -> Save. | * Change date in 008 with the new year, under Tools -> Manage Tasks -> Selected desired task in Task Lists window -> Manage Existing Tasks -> Edit Selected Task List -> Save. | ||
| Line 44: | Line 44: | ||
| * __Title fixes__: Find -> =245 -> Find All, and output to Excel. Screen titles for proper names (e.g. for people, countries, cities, scientific names, etc.) and for acronyms, and capitalize as required. //Hint: Information in the 520 field (Summary/Abstract) can be helpful; otherwise verify in SW.// | * __Title fixes__: Find -> =245 -> Find All, and output to Excel. Screen titles for proper names (e.g. for people, countries, cities, scientific names, etc.) and for acronyms, and capitalize as required. //Hint: Information in the 520 field (Summary/Abstract) can be helpful; otherwise verify in SW.// | ||
| * Make sure the non-filing character indicators are correct. (For example, a title beginning with a quote mark should be labeled 245 11.) | * Make sure the non-filing character indicators are correct. (For example, a title beginning with a quote mark should be labeled 245 11.) | ||
| - | * LAST STEP: MarcMake the MarcEdit file. This can be done by clicking File/Compile file into MARC with the .mrk document open in MarcEditor, or by closing the document and clicking the "Hammer & Wrench" MARC Tools icon in the MarcEdit home window. //Hint: If relabeling the file with an extension (i.e. .mrc), be careful when copying with "rename" in Services, to include the extension. I like to replace .mrk with mm, to avoid this problem.// | + | * LAST STEP: MarcMake the MarcEdit file. This can be done by clicking File/Compile file into MARC with the .mrk document open in MarcEditor, or by closing the document and clicking the "Hammer & Wrench" MARC Tools icon in the MarcEdit home window. //Hint: If relabeling the file with an extension (i.e. .mrc), be careful when copying with "rename" in Services, to include the extension. I like to replace .mrk with mm, to avoid this problem.// (Example): **umdissertations_septmm** |
| - | + | ||
| - | ** Upload to Connexion** | + | |
| + | ===Upload to Connexion=== | ||
| - **Prepare Local Save File** | - **Prepare Local Save File** | ||
| * __Option 1__: Go to File -> Local File Manager -> Create File. As follows: oai2019_dissertations, oai2019_theses, oai2019_thesesmfa, oai2019_theseslarp. (Connexion will add extension .bib.db) Highlight file just created, and Set as Default. Close. | * __Option 1__: Go to File -> Local File Manager -> Create File. As follows: oai2019_dissertations, oai2019_theses, oai2019_thesesmfa, oai2019_theseslarp. (Connexion will add extension .bib.db) Highlight file just created, and Set as Default. Close. | ||
| Line 57: | Line 56: | ||
| * Validate save file records: Highlight all records and hit Edit -> Validate. When finished, a report will be generated. Keep track of the record numbers reporting validation problems. (//Hint: I copy the entire report to Word or Notepad and delete all "Validation Successful" entries; if the remaining list is long, it can be printed for easy reference.//) Locate non-validated records by Save File number, open each one and fix the issue, then validate them singly. (**NOTE:** Most validation mistakes will be repetitive Field 653 key word entries, though sometimes something else will pop up, such as Chinese characters, see following step.) | * Validate save file records: Highlight all records and hit Edit -> Validate. When finished, a report will be generated. Keep track of the record numbers reporting validation problems. (//Hint: I copy the entire report to Word or Notepad and delete all "Validation Successful" entries; if the remaining list is long, it can be printed for easy reference.//) Locate non-validated records by Save File number, open each one and fix the issue, then validate them singly. (**NOTE:** Most validation mistakes will be repetitive Field 653 key word entries, though sometimes something else will pop up, such as Chinese characters, see following step.) | ||
| * To validate a record with Chinese characters, click Edit -> MARC-8 Characters -> Convert to MARC-8 CJK. Then Validate. | * To validate a record with Chinese characters, click Edit -> MARC-8 Characters -> Convert to MARC-8 CJK. Then Validate. | ||
| - | - **Super- and subscript, and Greek letter fixes:** | + | - **Super- and subscript, and Greek letter fixes:** OPTIONAL, if can be done without too much hassle!!! Not all sups, subs and symbols can be fixed; this is OK. |
| * These fixes can be done on the records in the Connexion Local File __after__ they are validated! A batch validation will not accept them, while they can be singly validated after the fixes are done. Most will be in the 520 Field (Summary/Abstract), though occasionally they will appear somewhere else, such as the title. OPTIONAL if can be done without too much hassle--some sups and subs and symbols cannot be fixed, e.g. if they are in the title. | * These fixes can be done on the records in the Connexion Local File __after__ they are validated! A batch validation will not accept them, while they can be singly validated after the fixes are done. Most will be in the 520 Field (Summary/Abstract), though occasionally they will appear somewhere else, such as the title. OPTIONAL if can be done without too much hassle--some sups and subs and symbols cannot be fixed, e.g. if they are in the title. | ||
| - | * Open the MarcEdit .mrk file, click Edit -> Find -> <sup -> Find All. Record record numbers where this is found; repeat with <sub, then with the common Greek letters (spelled out): alpha, beta, gamma, lambda, epsilon, mu. These will appear in the harvested records in brackets: [alpha] etc. | + | * Open the MarcEdit .mrk file, click Edit -> Find -> %%<sup ->%% Find All. Record record numbers where this is found; repeat with <sub, then with the common Greek letters (spelled out): alpha, beta, gamma, lambda, epsilon, mu. These will appear in the harvested records in brackets: [alpha] etc. |
| - | * Open the corresponding records in the Local Save File, and fix **<sup>2</sup>** (etc.) found there. Connexion will supply some sups and subs, found under Edit -> Enter Diacritics. Replace the entire **<sup>digit</sup>** string with the correct character. Word will supply a few more, found by opening Word, clicking Insert on the top bar, then Symbol/More Symbols, under Font: (normal text), by scrolling down to Superscripts and Subscripts under Subset: (Word sups and subs will properly transfer to Connexion, and will display in the Aleph OPAC with the correct Unicode.) Greek letters can be copied from Aleph, by pulling up a record and clicking the "brick wall" in the upper right corner. When the diacritic screen appears, choose Greek. //Hint: Search for "Greek alphabet" in Google, and refer to the pictorial representations for identification of the various letters.// | + | * Open the corresponding records in the Local Save File, and fix %%<sup>2</sup>%% (etc.) found there. Connexion will supply some sups and subs, found under Edit -> Enter Diacritics. Replace the entire %%<sup>digit</sup>%% string with the correct character. Word will supply a few more, found by opening Word, clicking Insert on the top bar, then Symbol/More Symbols, under Font: (normal text), by scrolling down to Superscripts and Subscripts under Subset: (Word sups and subs will properly transfer to Connexion, and will display in the Aleph OPAC with the correct Unicode.) Greek letters can be copied from Aleph, by pulling up a record and clicking the "brick wall" in the upper right corner. When the diacritic screen appears, choose Greek. //Hint: Search for "Greek alphabet" in Google, and refer to the pictorial representations for identification of the various letters.// |
| * After fixing sups/subs and Greek letters (and any other symbols easily found in Word, e.g. infinity sign), validate each record. | * After fixing sups/subs and Greek letters (and any other symbols easily found in Word, e.g. infinity sign), validate each record. | ||
| - **Update holdings/add OCLC#’s.** | - **Update holdings/add OCLC#’s.** | ||
| Line 68: | Line 67: | ||
| * Go to Tools -> Options -> Export. | * Go to Tools -> Options -> Export. | ||
| * Highlight File (Prompt for filename). Apply and Close. | * Highlight File (Prompt for filename). Apply and Close. | ||
| - | + | - **Export Local Save File** | |
| - | - **Export to Aleph** | + | |
| * Highlight all files in the Local Save File. Go to Action -> Export. | * Highlight all files in the Local Save File. Go to Action -> Export. | ||
| - | * Designate path and name for Output file: (example) U:\OAI\Dissertations\2019\umdissertations_septoclc. Exports as a .dat file. | + | * Designate path and name for Output file: (example) U:\OAI\Dissertations\2019\umdissertations_septoclc. Exports as a .dat file. (**NOTE:** The download will pause when non-AMA (term?) symbols are encountered. Note these numbers for fixing in Aleph later. |
| - | Switch to MarcEdit | + | ===Download into Aleph=== |
| - | 18. MarcBreak the file. | + | - **Preliminary MarcEdit Fixes** |
| - | MarcBreak this file. Replace .dat with .mb (or mb). | + | * MarcBreak the file. Replace .dat with mb. Open in MarcEditor. |
| - | 19. Change 049 field. | + | * Change AUMM to AUMETD. Go to File -> Edit -> Replace -> enter AUMM, AUMETD ->Replace All, Save. |
| - | Click Edit Records to get file into MarcEditor. | + | * Delete 035s (not needed here; records already have 001s). Go to Tools -> Add/Delete Field -> enter 035 into Field (no need supply data): -> Delete Field. Save. |
| - | Go to File Edit Replace all. Change AUMM to AUMETD. Save. | + | * File -> Edit -> Replace -> enter $zLink to resource, $zLink to free resource. Replace All, Save. |
| - | 20. MarcMake the file. | + | - **MarcMake the file** Replace mb with mm. (**NOTE:** This file should be named differently from the mm file loaded into Connexion (examples): **umdissertations_septoclcmm** vs. **umdissertations_septmm** Save to appropriate personal folder, and copy to FCL01/Scratch in WinSCP. |
| - | Click on File Compile File into MARC. This will save as a .mrc file. Save it to the right place, i.e. U:\OAI\Dissertations. OK to run .mrc through Services, but need to be careful to pick up the extension with rename/copy. Might be better to “lose” the extension! | + | - **Load records using Aleph Services** //Hint: Much time can be saved by Clicking **View History** and highlighting and opening the Service Form for the same jobs performed on earlier batches of materials.// |
| + | * Go to Services -> Load Catalog Records -> Advanced Generic Vendor Records Loader (File-90). Set Loader rules | ||
| + | - Input File name (example): umdissertations_septoclcmm | ||
| + | - Default Holding: AUMETD | ||
| + | - Character Conversion: OCLC_UTF_TO_UTF | ||
| + | - Fix Routine: UMFIX | ||
| + | - Match Routine: OCLC | ||
| + | - Merge Routine: OCLC | ||
| + | - Update Database: Yes | ||
| + | - Produce Loading Report: Yes | ||
| + | - Report file name: (example) umdissertations_sept2019report | ||
| + | * Add to History, Submit. | ||
| + | * Check results. When done (per Batch Log [A] under Task Manager), click [J] File List. The file name will appear under several versions: .failure, .single, .new and .multi. Highlight .new version (which should have the largest size, unless something glitched), make sure “Print Configuration” is set to “View HTML,” and click “Print” (to right of top window) to view the loaded records. Check one or two by bib number in the Aleph GUI to make sure they loaded correctly. An item and HOL should also have been created. | ||
| + | - ** LAST DETAILS: Globally remove 856 and add 910 fields from/to the bib records ** | ||
| + | * Go to WinSCP alephe/scratch to find the files for the newly-loaded records, under .adm, .bib, .hol, .items, and .orders. Use the .bib file, which will contain a Services-ready list of bib numbers. //Hint: I renamed this: (example) umdissertations_sept2019bibnos, and copied it to my personal folder for my records.// | ||
| + | * Go to Services -> Catalog Maintenance Procedures -> Global Changes (manage-21): | ||
| + | - Input file name: (i.e., umdissertations_sept2019bibnos) | ||
| + | - Output file name: (i.e., umdissertations_sept2019bibnos_del856) | ||
| + | - Line in Record -> Tag: 856; First Indicator: #, Second Indicator: # | ||
| + | - Delete Field – Yes. | ||
| + | - Add to History, Submit. | ||
| + | * Repeat this process to add a the 910: ABC 04/23/2020 BATCHN (ABC = your initials). This allows these records to be counted in the IRM monthly statistics. | ||
| + | - Input file name: (example) umdissertations_sept2019bibnos) | ||
| + | - Output file name: (example) umdissertations_sept2019bibnos_add910) | ||
| + | - Line in Record -> Tag: 910; leave indicators blank. | ||
| + | - Delete Field – No. | ||
| + | - Add to History, Submit. | ||
| - | Switch to Aleph | + | Job done! |
| - | 21. Load records | ||
| - | Copy .mrc file to FCL01/scratch in WinSCP | ||
| - | Go to Services Load Catalog Records Advanced Generic Vendor Records Loader FileFile (File-90). | ||
| - | 22. Set Loader rules (see snippet next page): | ||
| - | Input File name (for example, umthesesmay2016.mrc) | ||
| - | Default Holding – AUMETD | ||
| - | Character Conversion – OCLC_UTF_TO_UTF | ||
| - | Fix Routine – UMFIX | ||
| - | Match Routine – OCLC | ||
| - | Merge Routine – OCLC | ||
| - | Update Database – Yes | ||
| - | Produce Loading Report – Yes | ||
| - | Report file name (for example, umthesesmay2016_report) | ||
| - | Add to History, Submit. | ||
| - | + | -- //Contact person: [[lucyd@library.umass.edu| Lucy deGozzaldi]]// | |
| - | The process should now be complete. | + | |
| - | + | ||
| - | + | ||
| - | -- //Contact persons: [[kdion@library.umass.edu| Kay Dion]] or [[lucyd@library.umass.edu| Lucy deGozzaldi]]// | + | |
| - | * | ||
