LIGO Weekly Report for the week of 17 July - 21 July 2017
Past Weekly Reports
There will be no meeting of the Executive Committee this coming Monday
LSC recognized by the Royal Astronomical Society.
Martin Hendry, Mike Cruise and Francesco Pannarale accepted the RAS Group Achievement Award on behalf of the LSC.
David Shoemaker (Laura Cadonati)
Virgo determined that it can join the O2 run! The plan calls for joint observations starting in early August.
LIGO Laboratory Administration
Procurements and Subcontracts Management
Eric Garcia - CIT, LHO and LLO
*CIT- JPL (Researched discrepancies JPL PO)
*LLO- Battelle (Attempting to get additional information regarding their quote ongoing)
*LLO- NeoLASE (Draft SOW submitted to LLO and LHO team for review)
*LLO- LSU (Researched discrepancies with LSU regarding encumbrances requesting more info from LSULSU said it would take time because they are working on end of your stuff)
*LHO- SOW for Landscaping (in process submitted to Bubba for review)
*LHO- U of Oregon Mod. (in process)
*Both Sites- Mod VFA (in process)
*Executed some small dollar value procurements
Procurement Team Support
*Field phone calls related to various procurement questions for LHO, CIT, and LLO
*conducted follow up activities for several placed procurements
*Closeouts Resolved issues with closeouts
*executed tax overrides
*worked with Bailey to create a PowerPoint for the Quick Requisition Guide (draft complete)
Training Bailey Toon (on going)
Gina Salone CIT, LHO and LLO
*CIT- JPL (Researched discrepancies JPL PO)
*All Sites- completed a number of low dollar value procurements and punch-out purchases
*Completed a credit app to do business with a vendor
*Provided Document and file development support for Mods. From CIT, LHO, and LLO
Melanie McCandless - LLO
*LLO- NeoLASE (70W Builds in process)
*LLO- NeoLASE (PO Mod S303335)
*CIT- MP Aeor (SLiC Baffles and PCAL in process)
*LLO- LLO Took over follow up with ARS for termination from Eric (complete)
*All Sites- completed a number of low dollar value procurements
*Return items to McMaster Carr related to a punchout order
*Fields questions and provides procurement support to site end users
*Executed tax overrides
*Performed P-Card reconciliation
*Processed payment request for LSC fellow
Technical Writing Sevices
- FY18 AWP in progress
- Q3 report in progress
- Drop-in visitor outreach in Amber’s absence (5 people)
General Business Services
- Calendar updates: Travel calendar, LIGOMeetings
- Processing, reconciling Cardquest for Conf & Meetings card, Hiroto, Barish, Lazzarini
- Reconciling receipts/deposits for Amaldi
- Processing payments for Amaldi
- Diversity Meeting (minutes)
- Processing check requests
- Processing arriving visitors
- Updating Desk Assignments
- Updating for Staffing Committee Agenda page
- Handling image use queries.
- Processed expense reports.
- Allocated Pcard transactions.
- Answered inquiries from internal customers.
- Worked with external customers to facilitate travel accommodations.
- Updated CIT and LHO travel calendar.
- Processed visitors’ paperwork and submitted to appropriate department’s on campus.
- Tracked status of visitor’s through the approval process.
- Processed several payments for the LSC Activities fee.
- Issued several Payment Acknowledgments for the LSC Activities fee.
LIGO Laboratory Safety
Albert Lazzarini (Richard Oram, Bubba Gately)
The monthly meeting of the Safety Steering Committee was held this past Thursday. Minutes have been posted on the Laboratory Safety Web Page (access restricted).
Safety message for the week: Please take the time to report any unsafe condition that you may run across – and turn it into a Good Catch.
- Completed requested additional information for NSF 2018 AWP
- Worked with LLO GC team to move wireless PW from DCC to the shared secrets systems and socialized this to the LIGO users community
- Completed travel and workshop arraignments for NSF CyberSecurity Summit in August
LIGO Hanford Observatory (LHO) and Interferometer Operations
- G. Moreno, C. Romel, K. Ryan, J. Worden - Vacuum
- Bake oven VBOC is operational again with new equipment safety features; loaded chevron baffle parts for bake
- Baked and extensively reviewed AOSEM batch RGA scans and eventually passed load
- Ordered gate valves for main ion pumps for post O2 vent task
- Refining five year vacuum budget
- Designing chamber door hanger
- Installed spring tables for small ion pumps underneath cryopump 1 & 2
- Restocked VEA tool boxes with standard tools and added posted numbers on large gate valve bodies for easy identification
- Installed missing support tabs on viewport covers
- Editing viewport proof testing document E1100948
- Hunting down viewports for squeezer project and HAM 6 septum in site inventory
- Post O2 vent planning
- C. Carrasco, B. Gateley, K. Roland, C. Soike - Facilities
Hugh Radkins and Jim Warner
- Continued floor tilting studies, LHO alog 37479
- ISI Pod Pressure monitor development, LHO alog 37477
- Seismon EQ forecast tool, LHO alog 37497 & 37480 [JW]
- Violin mode damping training and doing, making violin mode
- PEM & SEI maintenance
- CPS Glitch investigation
- Post O2 activities prep.
- Spent time with Dave to understand and incorporate CDS list
of post-O2 tasks into the master LHO schedule E1700147, watching
for collisions of data and model down time against in-chamber
- Had a nice telecon with Aidan to work through incorporating
post-O2 TCS tasks into the E1700147 schedule - again watching
for other in-chamber collisions, and looking for potential
travel assistance for some tasks.
- Finalized first detailed draft of LHO Post-O2 vent schedule
E1700147-v9 - posted various views then presented at weekly SYS
meeting. Many questions asked and answered, leading to further
refinement of line items. Another revision pending!
- Sorted more parts with TJ to go to LLO for VOPO suspension
assembly. Located SUS PUM magnets to be used for VOPO
suspension. Sent some to LLO and MIT for their units.
- Sorted issues with Lisa regarding VOPO installation planning
and hardware availability.
- Worked through more ear fabrication questions from vendor.
- Verified stock of AERM and SRM parts (prisms, bump stops,
magnets) in the lab which need to be glued on in the coming
- Started digesting AERM prism gluing procedure and location
- Worked through some hardware availability issues for a SURF
- SLC weekly review/status mtg - walked through deliverables for
post-O2 vent plan.
- Coordinated with C&B for current and upcoming loads.
- Prepping Outreach talk for Star Party in Dayton, WA on Friday
- Getting a charge measurement post-Montana EQ LHO alog
- Calibration Measurement Suite
- Maintenance of DetEng FRS tickets 8540, 8539, 8517
- Testing & Commissioning tasks for O2 through O3 Break:
Dynamical model for VOPO suspension, Bounce / Roll Mode Damper
Testing, HAM-ISI Tuned Mass Dampers, and OFI SUS actuation
- Reset PSL after trip. See LHO alog 37560
and associated comments for full details.
- Trip was due to drop in flow in laser head 4.
- Unclear if flow drop was real or due to a glitch in the flow
- Work w/ B. Weaver on scheduling of LHO post-O2 PSL tasks.
- Work with PSL team to reconcile differences between the 2 LIGO
- Weekly PSL FAMIS tasks completed. See LHO alog 37579.
- Monitor health and performance of system.
- Laser head flows stable since 7/17/2017 laser trip. See LHO
- Continued work on laser SN 105–1.
- Monitor health and performance of system.
- Monitor health and performance of system.
Electronics, Control and Data Systems Operations
F. Clara, C. Perez, M. Pirello, Ed Merilh, G. Moreno
- Assisted with line hunting and causes at the end stations. Connecting monitors to the shields on the ESD drives. Also set up a test system in the H2 Building test stand to allow Pep to look for lines on the Beckhoff chassis and not interfere with operations. .
- Mounted all six squeezer slow controls chassis in the Test stand to allow software and hardware testing. Re-Worked the 24V DC distribution for better load separation. br>
- Working on Squeezing components looking for Junction boxes, N connectors and bulkhead feedthroughs for the Heliax RF cable runs.
- Relocated two racks for Squeezing. Put one in the CER and one in the LVEA by Ham 6. They are not in the final location as that has not been fully determined.
- Work continues on other squeezing hardware. Completed Frequency Discriminator boards for new TTFSS. Looking at the pre-amp for the TTFSS.
- Reviewing Optical Fiber Switch interface design.
- Working on lighting for OSB that is currently not covered by existing lights.
Control Room Operators
OPERATOR WEEKLY OVERVIEW
H1 continues to run at lower sensitivity due to Montana Earthquake from a few weeks ago & we run at a BNS Range of ~53Mpc. The cause of the range decrease is still under investigation.
OPERATOR OFF-SHIFT WORK
- Start getting quotes for HAM6 curtain
- Finish last of the parts collection for PSL Chiller modifications
- Work on implementing new gloves for site
- Standard administrative activities (CardQuest, etc)
- Continue ordering Squeezer Table Patch Panel hardware
- Going through inventory of Patch Panels & preparing shipment to LLO
- New Job Posting work
- FAMIS Tasks: 6157, 4534
- Control room support for PSL trip and IFO recovery on 7/17
- Assisted with installation work for the Beckhoff test rack
- Studying the calibration process for the Top Gun Ionizing Air Gun system
- alog 37584:IMC beam spot measurements - 17 July 2017
- alog37586:IO GigE camera installation on the PSL is complete
- alog37589: PSL IO GIGE cameras operational (Dave B)
- alog37595: comment to PSL IO GIGE camera’s operation
- Started tests of bolt torque on the H1 spare IM tower
Daniel Sigg (Keita Kawabe)
No commissioning due to O2.
H1 continues to run with higher noise than pre-earthquake with BNS range of ~55Mpc. Noise investigation continues.
Jitter coupling is there, but even after jitter subtraction, something else is hurting our performance between 20Hz and 100Hz (alog 37590).
LIGO Livingston Observatory (LLO) and Interferometer Operations
Observing Run #2
LLO achieved a weekly uptime performance (Weds-Weds) of just over 63% @ around 95 - 100 Mpc BNS range. Main causes of time loss this week were due to commissioning, locking acquisition, and corrective maintenance.
Recall that O2 commenced 10:00 am CT Nov 30th 2016. As of Friday, (33+ weeks or 5302 hours) into O2, Overall L1 operation has achieved a cumulative uptime of around 58.9%.
L1 cumulative uptime since the start of O2 is shown in the very useful performance charts
O2 summary of Observatory time accounting
Observing Time Loss due to HW/SW/ procedural faults
Summary of LLO Corrective Maintenance issues that have caused Observing Time Loss
We lost about 9 hours of observing time this week due to faults that required corrective maintenance.
8560 LLO Operations LLO Facilities 0.5 CLOSED DUPLICATE Power Glitch
8509 LLO Operations LLO Detector Enginee 8.5 WORKINPROGRESS — OMC locking trouble
70W Laser Amplifier
Following a field service Visit by NeoLASE the 35W Front End Laser was successfully realigned and was returned to nominal operating performance. This enabled the work on the NeoVan4s (70W) amplifier to continue.
See relevant alog entry:
The FE and amplifier combo are producing 70+ Watts (actually around ~79W) and is now being continuously run in LLO Laser Test Lab for the next few weeks to verify power stability. Laser workers continue to characterize the beam quality and pointing etc.
Observatory Computerized Maintenance Management.
FRS - Corrective Maintenance
Key Performance Indicator dashboard of issues tracked in the FRS.
Tuesday Maintenance Period Work Permits opened and or executed this week.
FAMIS- Preventive Maintenance
Note: LLO has shifted the maintenance period back by one hour. From now on it is scheduled from 9:00 am until 1:00 pm (local time) instead of 8:00 am until 12:00 pm.
Tuesday Work Permit Summary
LLO Tuesday Preventive Maintenance Work Orders 2017 (FAMIS)
Preventive Maintenance activities that were scheduled for execution during the month.
LLO now schedules and executes over 200 preventive maintenance work orders each month.
LHO now schedules and executes ~ 100 preventive maintenance work orders each month.
L1 Detection Coordination
The interferometer has been running well with a BNS range that often
exceeds 100 Mpc. Duty factor has been between 60% and 70% impacted
mainly by commissioning and some hardware problems.
LSC Fellows## (For the past 2 weeks)
Part of the time I have been looking at alog and summary pages and
participating in DetChar/DQshift meetings.
I wrote a matlab code to virtually rotate the seismometer and have
found 7.8 degrees in STS B.
I helped to install rubber feet under each of the power
racks in the corner power room.
I am helping DQ shifters by giving some information they ask for in
By studying the virtual rotation of the seismometers during an
earthquake, I found the same misalignment of axes result as previously
(in a high microseismic period).
I am studying the nature of ETMY saturations in an effort to establish a cause.
No report. Working on General Exam.
Repaired SHG circuitboard.
Electrically isolated diodes from the ISCT6 table.
Ordered connectors and cables for active components on ISCT6. Worked
with Carl + Michael on list of cables for inside the table enclosure.
Updated drawing of optical layout + spreadsheet of optical components
Set up SVN to share squeezer-related documents, figures, and code.
Started setting up to assemble + glue PZT mirrors for ISCT6.
Assembled, installed, tested PZT mirror on LO path on ISCT6
Started assembly of PZT mirror for seed path
Measured beam profile for in-vacuum fiber collimators for 532nm
Made temporary power/switch cable for fiber switch (Daniel S.
designing final version)
Set up GigE camera for SHG trans (with Joe B.)
Prep for cleaning in-vacuum optics (with Danny S.)
Round table discussion with SURF students
Arrived at LLO and settled down.
Picked up Terry’s work and finished the CLF path in the ISCT6.
Got familiar with the SHG and the green path on the IST6.
Rechecked the path of SHG to pump fiber, and changed the
Working on the ISCT6 table.
The problem with the green path was found and improved.
Working on the PSL, LO and PSLLO paths.
Waiting for some parts to arrive for measuring the fiber transmission.
Preparing for cleaning the optics in vacuum and squeezer platform.
Janeen Romie (Richard Oram)
Reporting for the LLO Detector Engineering Group: Stuart Aston, Matt Heintze, Adam Mullavey, Arnaud Pele
- Supporting interferometer alignment, debugging, commissioning and locking efforts. On-call 24/7 to help ensure maximum quality up-time of the IFO.
- With MIT visitor, Alvaro Fernandez Galiana, the VOPO suspension is assembled. The six AOSEMs were installed and driven. See TST alog 11314
- Performed PSL diode room and chiller room HVAC and make up air tests. Results may be found at alog 34899 for makeup air and alog 34968 for HVAC.
- Performed a full staging building HVAC turn off test to test ambient noise floor in the T&T laser lab. Details and accelerometer spectra may be found at alog 11365
- Supporting the SYS SLIC group, inventoried aLIGO dog clamps and cookie cutter suspension positioning fixtures. The inventory will be used to determine which of these items might be retrofitted for baffle installation after O2 ends.
- Performed PSL and TCS FAMIS tasks.
- Rerouting water in T&T lab after water leak in HPO, see T&T alog 11363.
- Gathering info on all the outstanding bugs, ECR's and other work to be done on the PSLs during the vent work tasks after O2. Kitting parts for 70W PSL install. Continuing tests on 70W amplifier in T&T lab, and working on list of proposed tests after moving amplifier into PSL in LLO and LHO.
- Gave visitor Benno Wilke a tour of the laser lab.
- Gave Caltech, LLO & UTRGV SURF students a tour of the LVEA and control room. Also participated in a round table discussion on the detector and graduate school.
- Working on vent plan; moving HAM6 work back to comply with squeezing plan. Adding TCS and PSL work. Working with LHO on vent, spares and 3rd IFO components.
Control Room Operators
Gary Traylor (Richard Oram)
Gary Traylor, Mike Fyffe, Doug Lormand, Joe Hanson, Danny Sellers, Jeremy Birch, Tom Evans, William Parker, Bryan Smith
Doug Lormand, Danny Sellers, Jeremy Birch, William Parker, Joe Hanson, Mike Fyffe,
- Site safety, security and data collection with continuous work schedule.
Assigned FAMIS work orders
Completed assigned FAMIS work orders
Adjusted the Operator work schedule to accommodate vacations and other issues
Assisted Tim Nelson in locating some of LIGO capital property inventory.
SRM optic prism bonding
Laser safety task
- 70 W Testing and Training Lab Water manifold troubleshooting & redesign
Computing and Electronics (CDS, etc.)
- Added puppet profiles to allow WEB, GC, and CDS LAN devices to share modules with only slightly different settings (i.e NTP server). Migrated more WEB LAN servers to puppet.
- Added fail2ban feature to permanently ban repeat offenders (ie ssh scanners)
- Built new GDS package with Keith Thorne’s awgstream timing, logging fixes.
- Created SL7.3 bootable thumb drives for simplified host installs
- Cleaned up some Nagios warnings after the recent migration to SL7
- Rebooted GigE camera server twice to recover from runaway camera processes
- Cleaned backup3 vtape 2
- Setup SL7 on new login portal in the DAQ test stand
- Switched over to the new SL7 nagios server this week and configured the contacts. Configured observium on nagios.
- Updated and rebooted one of the redundant file servers
DAQ test Stand:
- Completed, tested improved charge-measurement script for Stuart Aston
- Assisted SURF student with code development, model builds, data access questions
- Worked on real-time Linux patch changes to work with newer kernels
- Optics Lab test stand: Fixed mis-configured DAQ frame wiper scripts
CIT 40M Lab:
- Copied, shipped new boot server, framebuilder system disks to assist recover from hard disk failures
- Assisted Jamie Rollins with rebuilding models, drivers, etc.
- CDS Tuesday Maintenance tasks
- With Stuart Aston, verified that new charge-measurement scripts and awgstream changes eliminated failures
- Checked in seismon updates for earthquake monitor updates at LHO
- Debugged LDAS NDS2 server user login issues at LLO, LHO. Traced to failure of automated user list updating. Provided temporary list updates to resolve.
- Ordered material for control-room support and upgrades
- Outreach: Provided escort, transport to visiting SURF students from Pasadena
Carl Adams, Michael Laxen
Laser T&T Lab
- Control Box extra PD / PM channels
- Plumbing assistance
- Vibration isolation, Cables
Squeezed light install
- Assembly, Acceptance testing
- Writing test procedure
- Cable install planing and parts research and purchasing
- Dust Monitor install: RS485 node assembly completed
- RF mitigation: Planning post O2 measurements in PSL
- Troubleshooting SUS BSC DAC noise and loss of drive from DAC0. Removed and reinstalled test stand DAC.
- Rack Vibration mounts for corner-station DC Power Racks
- Vibration mounts for corner-stations SUS electronics racks
- Beam-Tube Monitoring: Testing non-contact IR thermocouple installed at one location
- Testing of all spare ADCs, DACs and adapter boards: Some need repair
Timothy Nelson (Richard Oram)
Completed FRS Update:
Projects and Enhancements
Cleaning is complete in the VPW (Vacuum Prep Warehouse). Cleaning was done with LLO staff to restore it to cleanroom status in preparation of upcoming clean and bake activities.
Facilities continues to share a staff member to assist the Vacuum Group with clean and bake operations.
Facilities assisted with July’s Science Saturday activities by providing staff to man the gate and control visitor access to the site in an effort to reduce the risk of breaking lock and loosing observing time.
Facilities assisted General Computing with the installation of multiple Wyfi access point around the site.
Facilities completed repairs to the all terrain, 6,000 pound capacity, forklift. The frame leveling hydraulic cylinder was leaking fluid badly making it unsafe to use. The cylinder was removed and repaired then reinstalled. Repairs were completed by LLO staff for a large cost savings and reducing down time.
Facilities completed repairs to the VPW 10 ton HVAC system with in-house staff avoiding an expensive service call and reducing down time.
Attended July’s Safety Committee Meeting
Facilities continues summer time PMs on all HVAC systems including condenser coil cleaning on systems for VPW, Staging Building, SEC, and other outlying buildings.
Facilities continues to complete all FRS requests and other daily duties including, custodial cleaning of the Corner Building, SEC, and Staging Building as well as technical cleaning in accessible garbing areas.
Facilities also completed routine FAMIS tasks including daily check and recording of the site potable water chlorine levels, weekly generator checks (3 total), plus other equipment checks like forklifts and manlifts.
Facilities continues to complete as many requests in house in order to save money including HVAC service and repair, electrical service and repair, plumbing service and repair, and general maintenance tasks.
Education and Outreach
- Conducted the monthly Science Saturday, when LLO opens its doors to the public. Facilitated by Kathy Holt, William Katzman, Amber Stuver and Tien Huynh-dinh.
- Conducted two tours for school students. Faciliated by Tien Huynh-Dinh, Kathy Holt and Amber Stuver.
- Facilitated visit from LIGO SURF students our of Caltech. Facilitated by Janeen Romie, Brian O’Reilly, Adam Mullavey, Keith Thorne, Tien Huynh-Dinh, Kathy Holt and William Katzman.
- Prepared for upcoming week long teacher workshop with teachers from Kansas Quarknet group.
Mechanics, Optics & Vacuum
- MOU review follow-up work.
- Organised and chaired LIGO diversity telecon.
- Looked at damping of Z baffles using viton washers - reduces Qs of peaks (Stephen, Calum, Norna).
- Checked info for AERM prism positioning (Garilynn, Stephen, Norna)
- Uploaded ECR for addition of actuation to the OFI (E1700226)
- STRAY light baffles: Started competitive procurement process for coatings of the baffles. Submitted RFQs to 3 DLC vendors and 1 Black Nickel vendor (Alena A. and Calum T.) Summer intern Ellie Parisi reported results on dust influence on scatter from AR coated black glass and DLC coating and developed a cleaning method for DLC coated super polished SSTL (G1701414)
- CRiMe: Annealed and inspected coated samples
Control and Data Systems
- Continuing squeezer hardware build. Summer Engineering Intern (Damian Lavin) continues to test the 384 channel Acromag Binary IO chassis.
- Luis is producing the final datasheet data for the PDH box he has produced and transitioning to working on the IO chassis upgrade.
- Todd has finished testing last remaining VOPO suspension Coil Drivers for Squeezer. He is now transitioning to AA board manufacture.
- Ordered (with help from Marie Woods) prototype electrometer circuit boards for in-vacuum electric field probe to be delivered to Rai.
- Working squeezer wiring library parts.
- Submitted in-vacuum wiring build to outside vendor for quote.
NDS2 (client, server, and server infrastructure)
- Fixed acl problems on the NDS2 servers
Data Analysis and Astrophysics LabWide Activities
- On family travel from Thursday through Monday.
- Work for APS DGRAV.
- Working on LIGO Lab Ops reports for FY17, annual work plan for FY18.
- Meet with SURFs Osase, Radha, Rico.
- Spend an inordinate amount of time defending the LOSC tutorial against charges that it contributed to the mistakes in Cresswell’s paper. It did not; there is nothing wrong with the tutorial, and if Cresswell had used it, they would have avoided many mistakes.
- Hosted 25 visiting undergraduate STEM students from Singapore (C.N. Yang Scholars). Gave them many hours of talks and tours (with help from Gabriele and Alex).
- Updated LOSC FAQ to include a “how-to” for adding events to the SVN.
- Discussed the issues with existing events in the LOSC SVN with Jonah and decided on the
path forward. Added GW170104 to SVN based on those discussions.
- Worked with Rory Smith and Paul Hopkins (Cardiff sysadmin) to test out a new module set
Paul put together there for developing with mpi4py and emcee packages under python at
- Identified a number of issues with the new module and am iterating with Paul as he is
- Reviewed a number of articles in circulation.
- Attended LSC working group meetings.
- Worked with Rory and Max on a project ranking background triggers using Bayes factors
- Investigated several instances of noise at LHO
- O1 CBC DQ paper passed P&P review
- Worked with two SURF students on their projects
- Fixed and updated LOSC summary pages
- Followed up loudest background events from analysis 16 and updated DQ flags
- Worked on analyzing the results of production CBC analyses
- O1 background runs for optimal astro background paper
- O1 background plus injection runs for same paper
- Working on draft of the same paper
- Working on draft of coherent bayesian detection statistic paper.
Advanced Distributed Data Analysis R&D
- Working at ARCCA to be able to run PE code under the Intel VTunes profiling environment.
A number of issues have been found and are being worked on their end.
LIGO Analysis and Open Data
Kent Blackburn (Jonah Kanner)
- Addressed a number of LOSC tickets.
- Discussed adding events to the SVN with Jonah and added GW170104.
- Updated the FAQ to include instructions for placing events in SVN.
- Worked with Juan and Stuart to arrange LOSC server updates.
LDAS Software Systems
- Worked on serialization of frameCPP library
- Worked on CMake conversion for ldastoolsal package.
- Worked on Portfile for latest version
LDAS System Administration
- Allocated additional dedicated nodes for Felipe
- Reconfigured the smc–7 switch after losing the configs
- Assisting Sharon with TFTP service access from the SMC switches
- Assisted Ryan with login access due to wrong shell in LDAP
- Updated the user provisioning script for creating the scratch dir
- Filed a bug report to E@H for transient HTTP error from the boinc clients
- Assisted Satya and Melody to restore LDAP and Grouper services
- Looked for cable tray options from Synchrotron
- Assisted Satya restoring ldap-test.ligo.org
- Cleared YUM cache on the cluster nodes to stop yum-cron e-mail errors
- Managed Grid-enabled user accounts
- Requested and installed IGTF, InCommon/Comodo and/or LIGO-CA SSL cert
- Modified “Ears and Prisms” website per Garilynn
Continued work rearchiving of T10KC -> T10KD (last 4 tapes from last
week finished verifying, started next batch).
O2 tape validation for last week finished without error.
LHO/LLO Hartman data up to date at CIT.
New clusterstage running now without trend frames. Has been running
fine since yesterday’s restart.
Got replacement HDD working in samarchive.
Fixed qfsnfs1’s 10.9 interface to use jumbo frames.
Created missing /scratch directories, gave Juan commands to add this
to the new user script.
Noticed that 40m data wasn’t getting to /hdfs…turns out it’s a
problem on the 40m end.
Set up Sharon on the x6-spare:/test (SSD mr) filesystem so she can do
some compile testing.
Did some work on the llcache script, but it needs more.
Put together rough list/$$ amounts for the 5 Year Plan.
- upgraded all Skylake switches to the latest firmware and downgraded CMMs to v1.99, bringing all switches and blades back online
- pulled a Broadwell node from the Condor pool for lldetchar testing
- removed PDUs from the Sun racks being sent to Caltech
- updated local account provisioning code to reflect recent changes to ldap.ligo.org
- LLO LDAS Account Activations (3)
Migrated all dedicated datanodes’ zfs kernel modules from dkms to kmod
Update to gds–2.18.0 on offline systems
Filed Oracle support tickets for: pci-e errors on sun-spare, sp/management service errors on most QFS servers.
Fixed broken Vulture installation on ldas-gridmon
Finished setting up standalone workstation in DCS room with Windows 10 VM for accessing ADMIN LAN devices, including Skylake node consoles.
Continued validating O2 data tapes
- DCC support: Changed document access levels, removed documents, made Amaildi documents public, add non-LSC authors and institutions.
- Support the Amaldi Meeting
- Worked on RT tickets:
- Tried to figure out how to resolve the PSP access error during the Grouper upgrade.
- Support for the MOU review: Looked at several issues such as author lists, answered several myLIGO questions.
- Spent time on on trying to recover data for 40 meter off a failed HD.
- Moving more equipment from my old office to my new office. "Much more work on this"
- Setup and updated GC software on a MBP to loan to a surf student.
- Move a few users to new offices, includes setting up network/power connection and moving/setting up all computers and other peripherals,
- Picked up equipment that was used in SCR for the MOU meeting. This equipment is put away.
- Added multiple mac addresses for visitors to our wireless access points.
- Ran OS/Security patches on NTSRV's & the Linux servers I maintain.
- Maintenance on main pdm server and also troubleshooted a backup/sync issue that was going on with this server.
- Added additional rule sets to the spam server to take care of users requests blocking spam/phishing email.
- Setup an additional monitor "TV" for an Engineer with a wireless HDMI setup for his visitors to look at when showing SolidWorks drawings etc. I had a request to rearranged his computers and monitors.
- Troubleshooted a wireless issue in W/B 3rd floor, Larry is now looking into issue something going on with the old network switch that is causing this issue.
- Some misc. onsite/remote/phone support for users.
- Amaldi12 web support.
- Work on the internal website.
- Work on the GWIC 3G website.
- Set up a google analytics account for Marc Favata.
- Ongoing web maintenance tasks.
- Picked up a couple of orders and setting up items for delivery to users.
Working on new quotes for maintenance contracts.
Tracking a couple of orders of new equipment.
Loaned out a number of computer support items to visitors.
Worked on a number of budget items for documentation.
- Finished up support for the Amaldi meeting at the hotel. Now cleaning up and storing items used at the conference.
Delivered a number of items that were transported back from the meeting, for other people.
Also, copying presentations from the presentation computers and sending them to David S. for archiving. Unfortunately, a number of presentations had already been removed from the systems.
Walked through some logistic issues for the MOU meeting, with Mike.
- Some minor work on moving of offices. Worked a number of logistical items for the next phase of the move.
- Patched a number of server and client boxes.
- Worked on the wireless problem in the SCR. One of the switches has a problem functioning in dual mode. A temporary work around has been put in place.
- Prepared things for cert update on one of the servers. The swap will be done after hours.
Made a number of changes on different service tables.
- Updated and started installing a few more support pkgs. on one of the new sandbox units. Hopefully, it will be ready for general testing at the end of next week.
- Some minor troubleshooting of some services, that had problems during a ldap glitch.
- Some testing and trouble-shooting of items on the ZFS appliances.
- Worked spam-filter appliance rules and updates.
- Regular user assistance and some assistance on one of the labs.
- Working on migrating user accounts to newer file server hardware.
- GC DNS configuration updates - replaced primary DNS servers with Debian systems, worked with ESnet to clean up reverse (in-addr.arpa) zone delegations.
- License renewals, parts procurements, hardware replacement planning.
- Work on PEM web server migration.
- Assist CDS with Puppet4 base setup for X1DTS along with “local” DNS configuration.
- Auditorium computer cleanups.
- Shibboleth IdP v3 installation/upgrade work (login3 basically complete, testing IDS software and auditd policy).
- Investigating KDC password policy vs. myLIGO implementation issues.
- Providing patch notices.
- Weekly patches checked/applied to servers.
- Security server maintenance, alerting/incident follow-up.
- Other misc. user support and daily sysadmin tasks.
- with Tyco tech, replaced guard shack Kantech controller with one pulled from VPW. Tyco rep ordered replacement for the VPW, and we installed it when it arrived. I ordered a spare for shelf. there will be no need for Tyco support for such work next time.
- more playing around with puppet 4.10 and hiera 5
- set SEC doors for Science Saturday, returned to work hours after
- user reported unable to import parts to ICS. permissions are set appropriately. investigated further by looking into log files and find no attempts by user to access the import page. user will followup upon return from vacation.
- in ICS, updated description on part records after user reported it to be incorrect
- in ICS, updated part name to remove invalid characters
- added VAC subsystem to ICS, updated import spreadsheet to include new subsystem, uploaded new revision to the DCC
- installed printer drivers for plotter on laptop
- re-imaged old Mac mini for shipping and receiving kiosk. installed printer/scanner so packing slips can be easily scanned and uploaded to FRS/DCC
- updated FRS to show classification when editing tickets.
- setup MPR for SURF round table discussion. worked audio during Teamspeak session of discussion with LHO.
- assisted Matt Cowart in replacing a few old wireless access points with new
- assisted Matt Cowart with Shibboleth/Apache settings on test sympa server
- purchased new version of parallels for MPR mac pro
- renewed GoToMyPC for facilities HVAC controls computer access
- spec’ing out new visiting engineer workstation
- updated FRS to include classification on all search results.
- worked a few other procurement items
- attended weekly meetings
- other misc user support
- Installed replacement wireless APs in the shared cubicle areas of the OSB building and the EE. The control room is next to be replaced
- Finished configuring and testing the backup sympa service for use to test patching and provide fault-tolerance
- Mitigation of e-mail problems generated by the attempted grouper upgrade
- Modified spam rules and scores to try and prevent false positives
- Tune up of new wireless WLANS
- Attended LIAM meeting
- Attended GC meeting
- Misc user requests
David Shoemaker (Stuart Anderson, Sharon Brunett, Peter Couvares)
Data Computing System (DCS)
- Quarterly Report submitted and accepted by NSF
- Approved RFQ for O3 data archive
Data Analysis Optimization
- prep work for the next 5y computing plan
- worked to enhance the LVC data analysis computing usage accounting system, identifying and reporting bugs, incorporating Open Science Grid computing usage, checking our calculation of each LIGO Data Grid site’s computing usage to accurately reflect the use of CPU Hyperthreading, and accounting for dedicated computing resources not managed by HTCondor.
- ad-hoc LIGO / Open Science Grid (OSG) systems administration duties
- LSC Computing & Software Committee chairperson duties (organizing and chairing meetings, preparing agendas, following up on requests and action items, etc.)
- prepared for the LIGO optimization advisory group meeting next month
- Continued work on PyCBC fused kernel development for GPU, and also avoiding a copy back to the CPU for the chi-squared calculation.
- Working on modifying test infrastructure to support multiple threads per core when benchmarking FFTs.
Data Analysis Optimization Operations supported