Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
batch_uploading_oais_to_oclc_and_aleph [2020/04/29 19:18]
ldegozzaldi
batch_uploading_oais_to_oclc_and_aleph [2022/05/16 18:53] (current)
jeustis
Line 1: Line 1:
-=====Batch Uploading OAIs from Scholarworks into OCLC and Aleph ======+===== PAGE OUTDATED ARCHIVED ​Batch Uploading OAIs from Scholarworks into OCLC and Aleph =====
 ==== CHANGE TITLE? ETDs (Current) Processing ScholarWorks OAIs ====  ==== CHANGE TITLE? ETDs (Current) Processing ScholarWorks OAIs ==== 
  
 NOTE:  My helpful "​hints"​ will appear in //​Italics.//​ NOTE:  My helpful "​hints"​ will appear in //​Italics.//​
  
-**Introduction**  +===Introduction===  
 The Graduate School will email "​Packing Lists" dated February, May and September (end of semesters) of new dissertations,​ theses, MFA theses and occasionally LARP theses. ​ There may be a lag between these dates and when the ETDs are available on ScholarWorks. ​ //I try to process them after a couple of months have passed, to assure that they will be picked up in the Crosswalk harvest.// The Graduate School will email "​Packing Lists" dated February, May and September (end of semesters) of new dissertations,​ theses, MFA theses and occasionally LARP theses. ​ There may be a lag between these dates and when the ETDs are available on ScholarWorks. ​ //I try to process them after a couple of months have passed, to assure that they will be picked up in the Crosswalk harvest.//
  
-**Preparation**+===Preparation===
   * Have a handy copy (either online or a printout) of the Packing List-in-process. **NOTE:** It's a good idea to save copies of these in appropriate folders. ​ Example: **PackingListReport_Feb2019diss.xlsx** in [Drive]:​\OAI\Dissertations\2019\ (i.e. 2019), or OAI\Theses, ThesesMFA or ThesesLARP.   * Have a handy copy (either online or a printout) of the Packing List-in-process. **NOTE:** It's a good idea to save copies of these in appropriate folders. ​ Example: **PackingListReport_Feb2019diss.xlsx** in [Drive]:​\OAI\Dissertations\2019\ (i.e. 2019), or OAI\Theses, ThesesMFA or ThesesLARP.
   * Open MarcEdit. ​ (**NOTE:** Make sure your MarcEdit XSLT engine is set to SAXON.NET. On MarcEdit home page, click tools(found on top), Preferences,​ MARCEngine, select SAXON/NET under XSLT Engine.)   * Open MarcEdit. ​ (**NOTE:** Make sure your MarcEdit XSLT engine is set to SAXON.NET. On MarcEdit home page, click tools(found on top), Preferences,​ MARCEngine, select SAXON/NET under XSLT Engine.)
  
-**Harvesting from ScholarWorks**+===Harvesting from ScholarWorks===
   - **Click on Harvest OAI Records:** (Found on either the MarcEdit home page or under (top) tools/OAI Harvester Tools/) Set the following:   - **Click on Harvest OAI Records:** (Found on either the MarcEdit home page or under (top) tools/OAI Harvester Tools/) Set the following:
     * Server address: https://​scholarworks.umass.edu/​do/​oai/​     * Server address: https://​scholarworks.umass.edu/​do/​oai/​
Line 27: Line 27:
     * **IMPORTANT NEW STEP, added 2020:**Go to ScholarWorks/​Dissertations and Theses and log onto "My account",​ scroll down to the appropriate series (i.e., DOCTORAL DISSERTATIONS (dissertations_2)/​Manage Dissertations/​Batch revise Excel/​Generate a spreadsheet of current data. See [[changing_one_year_campus_access_titles_to_open_access|Changing one year campus titles to open access in ScholarWorks]] for instructions on generating ScholarWorks spreadsheets. If extra names appear in the MarcEdit file, check the generated spreadsheet to make sure they are NOT dated in the range requested. //(This step has been added since occasionally a dissertation or thesis will have been left off the Packing List.)//     * **IMPORTANT NEW STEP, added 2020:**Go to ScholarWorks/​Dissertations and Theses and log onto "My account",​ scroll down to the appropriate series (i.e., DOCTORAL DISSERTATIONS (dissertations_2)/​Manage Dissertations/​Batch revise Excel/​Generate a spreadsheet of current data. See [[changing_one_year_campus_access_titles_to_open_access|Changing one year campus titles to open access in ScholarWorks]] for instructions on generating ScholarWorks spreadsheets. If extra names appear in the MarcEdit file, check the generated spreadsheet to make sure they are NOT dated in the range requested. //(This step has been added since occasionally a dissertation or thesis will have been left off the Packing List.)//
     * Any harvested record NOT on the Packing List that is also not on the generated spreadsheet,​ or has a different date (Check **degree_year** and **award_month**),​ or which belongs to a different series (such as English MFAs)can be removed from the MarcEdit file.     * Any harvested record NOT on the Packing List that is also not on the generated spreadsheet,​ or has a different date (Check **degree_year** and **award_month**),​ or which belongs to a different series (such as English MFAs)can be removed from the MarcEdit file.
-**Edit the MarcEdit ​.mrk file of harvested records**+===Edit the MarcEdit file of harvested records===
   - **Run MarcEdit task**   - **Run MarcEdit task**
     * Change date in 008 with the new year, under Tools -> Manage Tasks -> Selected desired task in Task Lists window -> Manage Existing Tasks -> Edit Selected Task List -> Save.     * Change date in 008 with the new year, under Tools -> Manage Tasks -> Selected desired task in Task Lists window -> Manage Existing Tasks -> Edit Selected Task List -> Save.
Line 44: Line 44:
     * __Title fixes__: Find -> =245 -> Find All, and output to Excel. Screen titles for proper names (e.g. for people, countries, cities, scientific names, etc.) and for acronyms, and capitalize as required. //Hint: Information in the 520 field (Summary/​Abstract) can be helpful; otherwise verify in SW.//     * __Title fixes__: Find -> =245 -> Find All, and output to Excel. Screen titles for proper names (e.g. for people, countries, cities, scientific names, etc.) and for acronyms, and capitalize as required. //Hint: Information in the 520 field (Summary/​Abstract) can be helpful; otherwise verify in SW.//
     * Make sure the non-filing character indicators are correct. (For example, a title beginning with a quote mark should be labeled 245 11.)     * Make sure the non-filing character indicators are correct. (For example, a title beginning with a quote mark should be labeled 245 11.)
-    * LAST STEP: MarcMake the MarcEdit file. This can be done by clicking File/​Compile file into MARC with the .mrk document open in MarcEditor, or by closing the document and clicking the "​Hammer & Wrench"​ MARC Tools icon in the MarcEdit home window. //Hint: If relabeling the file with an extension (i.e. .mrc), be careful when copying with "​rename"​ in Services, to include the extension. ​ I like to replace .mrk with mm, to avoid this problem.// +    * LAST STEP: MarcMake the MarcEdit file. This can be done by clicking File/​Compile file into MARC with the .mrk document open in MarcEditor, or by closing the document and clicking the "​Hammer & Wrench"​ MARC Tools icon in the MarcEdit home window. //Hint: If relabeling the file with an extension (i.e. .mrc), be careful when copying with "​rename"​ in Services, to include the extension. ​ I like to replace .mrk with mm, to avoid this problem.// ​(Example): ​**umdissertations_septmm**
- +
-** Upload to Connexion**+
  
 +===Upload to Connexion===
   - **Prepare Local Save File**   - **Prepare Local Save File**
     * __Option 1__: Go to File -> Local File Manager -> Create File.  As follows: oai2019_dissertations,​ oai2019_theses,​ oai2019_thesesmfa,​ oai2019_theseslarp. (Connexion will add extension .bib.db) Highlight file just created, and Set as Default. Close.     * __Option 1__: Go to File -> Local File Manager -> Create File.  As follows: oai2019_dissertations,​ oai2019_theses,​ oai2019_thesesmfa,​ oai2019_theseslarp. (Connexion will add extension .bib.db) Highlight file just created, and Set as Default. Close.
-    * __Option 2__: If working in the same year, go to Cataloging -> Search -> Local Save File. Click the drop-down arrow at right end of "Local File" space, choose the file for dissertations,​ etc. under the correct year. Hit OK to openautomatically ​setting ​it as the default file. Highlight all records currently and hit Action -> Delete. Screen will be blank.+    * __Option 2__: If working in the same year, go to Cataloging -> Search -> Local Save File. Click the drop-down arrow at right end of __Local File__ window, choose the file for dissertations,​ etc. under the correct year, and hit "OK" ​to open it up. This will automatically ​set it as the default file. Highlight all records currently ​in the file, and hit Action -> Delete. Screen will become ​blank.
   - **Import records from MarcEdit**   - **Import records from MarcEdit**
-    * Go to File -> Import Records... Browse for correct mm (or .mrc) file to enter into "​File ​to Import.+    * Go to File -> Import Records... Browse for correct mm (or .mrc) file to enter into __File ​to Import____Destination__ ​= Import to Local Save File. __Bibliographic__ ​= appropriate Local Save File (set as default). (**NOTE:** Character Set under __Record Characteristics__/​Bibliographic Records needs to be UTF-8 Unicode.) ​Hit OK; close Report window. 
-    *   +  - **Manage records in Connexion Local Save File** 
-Browse for mm file to enter into “File to Import” field. +    ​* ​Go to Cataloging ​-> Search ​-> Local Save File. Correct Local Save File should appear in top field. Hit OK. 
-Destination=Import to Local Save File +    * Validate save file records: ​ Highlight all records and hit Edit -> ValidateWhen finished, ​report will be generatedKeep track of the record numbers reporting validation problems. (//Hint: I copy the entire report to Word or Notepad and delete all "​Validation Successful"​ entries; if the remaining list is long, it can be printed for easy reference.//) Locate non-validated records by Save File number, open each one and fix the issue, then validate them singly(**NOTE:** Most validation mistakes will be repetitive Field 653 key word entries, though sometimes something else will pop up, such as Chinese characters, see following step.) 
-Bibliographic=appropriate Local Save File (set as default) +    * To validate a record with Chinese characters, click Edit -> MARC-8 Characters -> Convert to MARC-8 CJKThen Validate.  
-Hit OK; close Report window. +  ​- **Super- and subscript, and Greek letter fixes:** OPTIONAL, if can be done without too much hassle!!! Not all sups, subs and symbols can be fixed; this is OK. 
-12. Manage records in Connexion Local Save File. +    * These fixes can be done on the records in the Connexion Local File __after__ they are validated! A batch validation will not accept them, while they can be singly validated after the fixes are done. Most will be in the 520 Field (Summary/​Abstract),​ though occasionally they will appear somewhere else, such as the title. OPTIONAL if can be done without too much hassle--some sups and subs and symbols cannot be fixed, e.g. if they are in the title. 
-Go to Cataloging ​ Search ​ Local Save File.   +    * Open the MarcEdit .mrk file, click Edit -> Find -> %%<sup ->%% Find All.  Record record numbers where this is found; repeat with <sub, then with the common Greek letters (spelled out): alpha, beta, gamma, lambda, epsilon, mu. These will appear in the harvested records in brackets: [alpha] etc. 
-Correct Local Save File name should appear in top field. ​ Hit OK. +    * Open the corresponding ​records ​in the Local Save File, and fix %%<​sup>​2</​sup>​%% (etc.) found there. Connexion will supply some sups and subs, found under Edit -> Enter DiacriticsReplace the entire %%<​sup>​digit</​sup>​%% string with the correct character Word will supply a few more, found by opening Word, clicking Insert on the top bar, then Symbol/More Symbols, under Font: (normal text), by scrolling down to Superscripts and Subscripts under Subset: (Word sups and subs will properly transfer to Connexion, and will display in the Aleph OPAC with the correct Unicode.) Greek letters can be copied from Aleph, by pulling up a record and clicking the "brick wall" in the upper right corner. When the diacritic screen appears, choose Greek. //Hint: Search for "Greek alphabet"​ in Google, and refer to the pictorial representations for identification of the various letters.// 
-13. Proofread author names against Packing List as last double-check. +    * After fixing sups/subs and Greek letters (and any other symbols easily found in Word, e.g. infinity sign), validate each record. 
-Hint:  ​Alphabetize author names ​Verify that names match and correct any errors+  - **Update holdings/​add OCLC#’s.** 
-14. Validate ​file records+    * Be sure to log onto Connexion. Highlight ​records ​and click the green Update arrow in the top bar, under the word "Batch." ​Wait for it to stop "ticking.
-Hint:  Return file to numerical order (This will make corrections easier!+    * Double-check OCLC# column for blanks (missed validation). ​//Hint: Sort by clicking the heading, Control #, afterward returning the list to its original order by clicking Save #.//  ​Validate and update ​any blanks found
-Highlight all records and hit Edit ValidatePrint resulting list if long+  - **Set Connexion Export parameters.** 
-15. Update holdings/​add OCLC#​’s. +    ​* ​Go to Tools -> Options ​-> Export. 
-Log on to Connexion, highlight ​recordsclick Update arrow. ​ Wait for it to stop ticking.”  ​Double-check OCLC# column for blanks (missed validation). ​ Validate and update ​these+    ​* ​Highlight File (Prompt for filename). Apply and Close.  
-16. Set Connexion Export parameters. +  - **Export ​Local Save File** 
-Go to Tools  Options ​ Export. +    ​* ​Highlight all files in the Local Save File.  Go to Action ​-> Export. 
-Highlight File (Prompt for filename). ​ Apply and Close. +    ​* ​Designate path and name for Output file: (example) ​U:​\OAI\Dissertations\2019\umdissertations_septoclc. Exports as a .dat file. (**NOTE:** The download will pause when non-AMA (term?) symbols are encountered. Note these numbers for fixing in Aleph later.
-17. Export ​to Aleph +
-Highlight all files in the Local Save File.  Go to Action ​ Export. +
-Designate path and name for Output file:  ​i.e., ​U:\OAI\ Dissertations\umdissertationsMay2016. Exports as a .dat file.+
  
 +===Download into Aleph===
 +  - **Preliminary MarcEdit Fixes**
 +    * MarcBreak the file. Replace .dat with mb. Open in MarcEditor.
 +    * Change AUMM to AUMETD. Go to File -> Edit -> Replace -> enter AUMM, AUMETD ->​Replace All, Save.
 +    * Delete 035s (not needed here; records already have 001s). Go to Tools -> Add/Delete Field -> enter 035 into Field (no need supply data): -> Delete Field. Save.
 +    * File -> Edit -> Replace -> enter $zLink to resource, $zLink to free resource. Replace All, Save.
 +  - **MarcMake the file** Replace mb with mm.  (**NOTE:** This file should be named differently from the mm file loaded into Connexion (examples): **umdissertations_septoclcmm** vs. **umdissertations_septmm** ​ Save to appropriate personal folder, and copy to FCL01/​Scratch in WinSCP.
 +  - **Load records using Aleph Services** ​ //Hint: Much time can be saved by Clicking **View History** and highlighting and opening the Service Form for the same jobs performed on earlier batches of materials.//​
 +    * Go to Services -> Load Catalog Records -> Advanced Generic Vendor Records Loader (File-90). Set Loader rules
 +        - Input File name (example): umdissertations_septoclcmm
 +        - Default Holding: AUMETD
 +        - Character Conversion: OCLC_UTF_TO_UTF
 +        - Fix Routine: UMFIX
 +        - Match Routine: OCLC
 +        - Merge Routine: OCLC
 +        - Update Database: Yes
 +        - Produce Loading Report: Yes
 +        - Report file name: (example) umdissertations_sept2019report
 +    * Add to History, Submit.
 +    * Check results. When done (per Batch Log [A] under Task Manager), click [J] File List. The file name will appear under several versions: ​ .failure, .single, .new and .multi. ​ Highlight .new version (which should have the largest size, unless something glitched), make sure “Print Configuration” is set to “View HTML,” and click “Print” (to right of top window) to view the loaded records. ​ Check one or two by bib number in the Aleph GUI to make sure they loaded correctly. An item and HOL should also have been created.
 +  - ** LAST DETAILS: Globally remove 856 and add 910 fields from/to the bib records **
 +     * Go to WinSCP alephe/​scratch to find the files for the newly-loaded records, under .adm, .bib, .hol, .items, and .orders. ​ Use the .bib file, which will contain a Services-ready list of bib numbers. ​ //Hint: I renamed this: (example) umdissertations_sept2019bibnos,​ and copied it to my personal folder for my records.//
 +     * Go to Services -> Catalog Maintenance Procedures -> Global Changes (manage-21):​
 +       - Input file name: (i.e., umdissertations_sept2019bibnos)
 +       - Output file name: (i.e., umdissertations_sept2019bibnos_del856)
 +       - Line in Record -> Tag: 856; First Indicator: #, Second Indicator: #
 +       - Delete Field – Yes.
 +       - Add to History, Submit.
 +     * Repeat this process to add a the 910:  ABC 04/23/2020 BATCHN (ABC = your initials). This allows these records to be counted in the IRM monthly statistics.
 +       - Input file name: (example) umdissertations_sept2019bibnos)
 +       - Output file name: (example) umdissertations_sept2019bibnos_add910)
 +       - Line in Record -> Tag: 910; leave indicators blank.
 +       - Delete Field – No.
 +       - Add to History, Submit.
  
 +Job done!
  
  
- +-- //​Contact ​person: [[lucyd@library.umass.edu| Lucy deGozzaldi]]// ​     ​ 
- + 
-After importing the bib records file from MarcEdit  +
- +
-(For example purposes, we will use the Connexion file for February 2016 Dissertations which can be opened via CatalogingSearchLocalSaveFile -> <​nowiki>​T:​\\oclcapps\Connexion\Theses\2016_Feb_Dissertations.bib.db)</​nowiki>​) +
- +
-  * Highlight all records in the file and Validate (Edit -> Validate or Shift+F5). This will generate a report of results. Note which records did not validate and make the necessary corrections. Re-validate as needed.  +
-  * Highlight all records in the file and Update Holdings (Action -> Holdings -> Update Holdings or F8). OCLC record numbers will begin appearing in the file as each record is uploaded.  +
- +
-**To export from Connexion to Aleph:** +
- +
-  * Go to Tools -> Options and click on the Export tab. Highlight the __Prompt for filename__ option then check off the box for Display report for immediate export results. Click on Apply then Close. +
-  * Open the Local Save file you want to export (2016_Feb_Dissertations - See path above) +
-  * Highlight records +
-  * Export (Action - Export or F5) +
-    This will ask where to put the output file in your C: drive and what name to use. Make sure the filename is in all __lower case__ - for example, feb2016diss. The file will be downloaded into your C: drive as a .dat file. (Example: C:​\Crosswalk\Dissertation&​Theses\Connexion_Records\feb2016diss.dat) +
-  * Open MARCTools in MarcEdit. +
-  * Input the .dat file from your C: drive (feb2016diss.dat)and name the Output file with a .mb extension (feb2016diss.mb. Execute the MarcBreaker. +
-  * Click on Edit Records. Use Replace to change AUMM to AUMETD. +
-  * Under MARCEditor --> File, click on Compile File into Marc. This will save as a .mrc (MARC) file.  +
-     +
-  * Open Aleph, Cataloging function +
-  * Click on Task Manager then [F] Upload/​Download files +
-  * Find where your saved .mrc file is on your C: drive (feb2016diss.mrc) and copy to the FCL01/​Scratch file (from drop-down menu over left Remote Files column)by clicking on the left arrow button between columns +
-  * In the Aleph menu bar above, click on *_Services -> Load Catalog Records +
-  * Click on Advanced Generic Vendor Records Loader (file_90) +
-    Make sure the following rules are set: +
-     * Input File name (for this example, feb2016diss.mrc) +
-     * Default Holding - AUMETD +
-     * Character Conversion - OCLC_UTF_TO_UTF +
-     * Fix Routine - UMFIX +
-     * Match Routine - OCLC +
-     * Merge Routine - OCLC +
-     * Update Database - Yes +
-     * Produce Loading Report - Yes +
-     * Report file name(for this example, feb2016_report) +
-     * Click on the Submit button at top right +
- +
-Once the exporting is done, click on Task Manager -> [A] Batch Log to view the report. +
-     * Highlight your file (p_file_90) and click on View Printouts.  +
-     * Under Remote Name, highlight <​filename>​_report.new (i.e. feb2016diss_report_new) +
-     * Click on Print to obtain reports. ​ You want the loader-log-report which will show the FCL01 Bib Sys numbers for each record. ​ Copy one and check the bib record which displays for any potential corrections needed.  +
- +
-**To Globally Remove the 856 Field from Bib Records:​** +
- +
-  * Click on *_Services -> Catalog Maintenance Procedures -> Global Changes (manage-21) +
-   Set the rules: +
-  * Input file name <​filename>​.mrc.bib (i.e., feb2016diss.mrc.bib) +
-  * Output file name <​filename>​.mrc856 (i.e., feb2016.diss.mrc856) +
-  * Update Database - Yes +
-  * Line in Record -> Tag -> 856; first indicator - #  second indicator - # +
-  * Delete field - Yes +
-  * Click on Submit button +
- +
-The process should now be complete.  +
- +
- +
--- //​Contact ​persons[[kdion@library.umass.edu| Kay Dion]] or [[lucyd@library.umass.edu| Lucy deGozzaldi]]// ​     ​+
    
- 
                    
            
  
batch_uploading_oais_to_oclc_and_aleph.1588187928.txt.gz · Last modified: 2020/04/29 19:18 by ldegozzaldi
[unknown link type]Back to top
www.chimeric.de Creative Commons License Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0