PCPS Field Service Engineer 4
Field Service Engineer (FSE-4) - Equipment Integration
-
I had never worked on Alcatel equipment, but I was hired because I knew how to use the TL1 language. The first product that I worked on was the 1630SX DACS. Having never worked on this equipment, I was given the PDF manual set which I studied. Within a month, I passed the Alcatel Field Force Certification test and troubleshooting hands-on and written exams. I was listed as a FSE4 performing test and turn-up tasks on live traffic carrying systems such as: Rack additions, Shelf additions, live Shelf replacements, Troubleshooting and resolving card errors, RMA procedures. Sometimes there were errors on the new or used equipment. By performing tests with TL1 commands, T1/T3 test sets and problem solving skills I was able to isolate the error(s).
-
I continued to learn other Alcatel equipment: 1631LMC, 1677 SONET LINK, 1671 Service Connect and the 1696 WDM, performing integration and hardware/software upgrades.
-
I performed these tasks on hundreds of projects with only two issues over the course of 4 years:
-
One issue was a card switch discovered during system testing prior to performing a major upgrade. I had noticed that the card was switched to protect the night before and I had alerted the site manager of the issue and suggested that the card be replaced. I noted the issue on my daily report. The system had many other errors prior to my arrival. New cards were failing ("out of box failure" reporting MRE errors) when inserted into the system. There were already over a dozen failures prior to my arrival. I inspected the shelves where the cards were failing. I discovered that there was an excessing amount of dust in the backplane. (The site was having building upgrades and the Air Control Unit was supplemented with portable cooling units.) The air-filters hadn't been cleaned during the building process and dust was escaping around the filters contaminating the backplane. I was able to clean the back planes using compressed air and an ESD safe vacuum. I re-inserted cards that had already been marked as failed. The cards were able to initiate properly and performed tests without errors. After resolving the dust issues, I informed upper management of the issue and that many of the returned cards were probably not "out of box" failures. However, when it came time to continue my designated work and perform a matrix switch test, the card that I had identified previously, performed a switch to protect. Even though I was in a valid maintenance window, since the card was carrying live traffic, I was required to complete FCC reports. I should have verified the part number of the card to insure that it had been replaced instead of just being switched back to "working" by the site manager. Replacing the card with a new unit corrected the issue. I consider this my error for not checking the card's part number. The rest of the project was completed without any issues.
-
The second issue was with an EC (Equipment Controller) card. I was performing a software upgrade on the shelf. I was alerted that the system seemed to be responding slowly to commands, but traffic was unaffected. The normal software upgrade as per the Customer Release Notes, would cause minor traffic hits to the DS3 circuits. I performed the software upgrade on a non-traffic carrying shelf first. It went without issue. I then started the software upgrade on the live shelf. I noticed that the procedure was not running as fast as normal. I contacted TAC and after stopping the software upgrade it was decided to restart the upgrade and let it finish even though it was running slowly. I diligently kept track of the progress. I started to suspect that the upgrade would not complete during the maintenance window even though it should have completed with over an hour to spare. I calculated that the back-out procedure also would not complete within the maintenance window. There was no turning back at this point. I alerted TAC and the site tech of the potential DS3 switch that would take place approximately ½ hour after the end of the maintenance window. It was determined that both EC cards in the system were having issues and were later replaced. Even though TAC suggested that we proceed, I was the technician on site and I should not have allowed a restart of the upgrade. I accepted responsibility for the DS3 switch outside of the maintenance window.