We had outgrown our IP allotment and it was time to upgrade. For the first 23 years of VECC we had enough IP addresses to do the things that we needed. We had used up our 254 pubilc IP’s. We didn’t own our own IP’s they were leased from Xmission. We found out through Century Link that VECC was entitled get more public IP’s through a program that had been set aside for public safety. We quickly filled out the paper work and got two class C public address ranges. This was more than enough to do what we had in mind.
When I first started at VECC the software that we used required public IP addresses now this was no longer the case. We could now go to private addressing and VLAN out our networks. I was tasked with making this happen for VECC.
The goals were simple.
Change all the IP addresses in our network to private including Windows and Unix servers
Create VLANs to segregate our different needs
NAT our private server addresses to public IPs.
Make all the necessary routing changes at VECC and coordinate all the routing changes with our different agencies.
Update our two Cisco ASA firewalls to the latest software.
Install and configure Cisco’s AnyConnect VPN
Less than an hour of down time.
I was able to accomplish all of this in just a couple of weeks. I coordinated with all the different VECC agencies to make the switch on our Frame Relay connections, our Cisco to Cisco IPSec connections as well as our basic internet routing.
I created 5 VLANs for our various needs like servers, workstations, web servers, wireless internal and external. I configured all of our switches and the appropriate ports for the VLAN. I mapped out the entire physical network, something that had never been done at VECC ever.
This project was a huge undertaking due to all the different agencies that connect to VECC. My boss credited me with it being a huge success.
Navigator has become one of the premiere 9-1-1 dispatch conferences. This year’s conference was held in Salt Lake City, Utah. I was asked by a coworker to present at this years National Academies of Emergency Dispatch’s Navigator Conference. My coworker has presented several times and thought I would be a good fit because of my passion for the subject. We called this years presentation, “5 Star Training on a Dime Budget”.
Most 9-1-1 centers have very slim budgets and large training requirements. My employer VECC has allowed me to implement many freeware solutions. Our presentation focused on a few freeware products that have made a huge difference in our training programs.
Here is an overview of our presentation.
Road blocks – People and resources
–Bridge it or work around
Road blocks will spring out of every nook and crevice. Be prepared to bridge it or work around it.
You will have people come out of the woodwork who will surprise you by either being a champion of your project or a roadblock. Be prepared to over come the fear of change.
Learn about your data network. Learn about what you have and what can be done.
Learn about what assets you have to work with.
Find an IT person to champion your idea.
Back-end People and Front-end People.
What do you need? Support for Hardware and Software. Is your IT person a member or your organization or an outsider (integrated/separate/outsourced)
Support for users. Training, Upkeep, A champion.
You will need to assemble a team of people from IT to End User.
1. MediaWiki and the VECCipedia
Veccipedia has been a super successful implementation of a wiki that is used for documentation and training. You can read more about it here.
We showed how we used a wiki as our training hub.
2. Google Docs
VECC has been using google docs for more than 5 years as a way to share information, training and scheduling. We also included Google Drive
Claroline is a classroom training software we use to create training programs. It includes career paths, testing and training.
Moodle is another classroom training software. 9-1-1 employees don’t get to experience much offline training. They usually are forced to self train at their desks.
Camstudio is free software we use to do screen capture training. It allows us to record, annotate and record audio of a computer desktop. We use this a lot in training on Computer Aided Dispatch (CAD) Software
6. Microsoft Office
My co-presenter had built a master piece of tracking DOR’s on trainees using Excel spreadsheet.
I think our presentation was successful. We had a standing room only crowd of more than 200. We received excellent feedback and audience participation. For me the highlight was when several people in the crowd had mentioned that they had visited VECC and had been inspired by our Wiki and went back to their agencies and created their own.
My team, Technical Services at Valley Emergency Communications Center has spent much of the last year researching, funding, planning, installing and implementing a revolutionary IP based 9-1-1 call delivery system in Utah. Our idea was so revolutionary and groundbreaking as well as being so cost effective that several other agencies asked to join in our system with many others planning on joining in the near future.
9-1-1 call centers need to start rolling out NG 911 or Next Generation 911 by early next year as mandated by the Federal Government. NG-9-1-1 is designed to handle text to 911 as well as video or pictures. These features are currently not active with our new system so don’t try texting 911 just yet. The new phone system gives us the flexibility to offer those services in the very near future.
The new phone system was a third the price of a typical 9-1-1 call delivery system saving potentially millions of dollars in 9-1-1 funds.
The install was accomplished over a 48 hour period. We divided our land lines and cell lines between the two system and brought up one position at a time. It was a monstrous project completed with co-operation and hard work from VECC IT staff , VECC operations , Intrado, Vectra Solutions, and Alpine Geographic.
Agencies, vendors deliver the state’s first emergency services Internet protocol network (ESInet) for 9-1-1 calls
9-1-1 call centers serving Salt Lake, Davis, Weber and Morgan counties have taken the initiative to provide the first Internet protocol (IP)-capable 9-1-1 call delivery system in Utah. Salt Lake Valley Emergency Communications Center (VECC) Executive Director, William Harry, State of Utah Department of Public Safety (DPS)/Salt Lake Communications Center (SLCC) Dispatch Center Manager, Chris Rueckert and Weber Area Dispatch 9-1-1 and Emergency Services District (Weber) Executive Director, Tina Scarlet, have led this collaboration of six individual public safety agencies to provide multiple levels of 9-1-1 network survivability—ultimately improving public safety for approximately 1.6 million citizens.
In November 2011, the team came together to submit a request for information for an IP-based, multi-node, 9-1-1 call-handling solution. They wanted to implement a “shared model” as many of the various public safety answering points’ (PSAPs) call-processing equipment (CPE) was approaching end of life and would no longer be supported. The project, “The Greater Wasatch Multi-Node Project,” was established to find an IP-delivery system that would provide multiple levels of redundancy, advanced call-routing capabilities, greater efficiencies and ultimately less cost than their various on-premises solutions.
The plan for the Greater Wasatch Multi-Node Project was to use the VECC and Weber 9-1-1 centers to host the CPE and call-taking applications, providing access to other PSAPs via the ESInet. This plan would allow PSAPs to pool their resources by sharing costs for two centers instead of spending funds on CPE and applications for multiple centers.
When the collaborating entities compared their current vendor’s solution to alternatives offered by Intrado and CenturyLink, they decided that the Intrado and CenturyLink platform had the IP capabilities that best met their expectations. Intrado and CenturyLink offered an emergency service number (ESN)-based ESInet for all 9-1-1 wireline and wireless call handling and routing; IP-based Intrado VIPER® call processing equipment that provided quadruple redundancy at VECC and Weber; and a common database for call routing and mapping systems that display caller location.
Once the team made their decision, the Utah 9-1-1 Committee funded most of the project with grants.
In January of 2013, VECC, Weber Area 9-1-1, and DPS/SLCC went live on the new ESInet, surpassing expectations as the first IP-capable 9-1-1 network for the state of Utah. As a result, the Greater Wasatch Multi-Node Program has provided immediate financial benefits. “This new shared model has already provided substantial savings on both CPE and call-trunking costs,” explained Rueckert.
As an added feature, the solution also provides “agent roaming.” In other words, the new platform can leverage advanced call-routing technology to share call-taking positions from call takers across different PSAPs during times of high-call volume. “A 9-1-1 call taker or dispatcher can log into any shared workstation, at any location, and receive and dispatch emergency calls as if they were at their own PSAP,” said Scarlet. “This collaboration of individual PSAPs and sharing of resources is unprecedented in Utah.”
“The sharing of resources is a new frontier for all of us,” said Rueckert. “All too often, we see other state governments mandate that PSAPs collaborate and create a consolidated solution. We took this initiative on ourselves, because it was the right thing to do.”
Other PSAPs across the state are lining up to benefit from this new shared-network solution. The Bountiful City Police Department, Greater Salt Lake Unified Police Department and the Salt Lake City Police Department are just a few agencies that will be joining the Greater Wasatch Multi-Node Program in the very near future. “The new ESInet platform started as a beta test for NextGen 9-1-1 and now it’s become the standard for new deployments,” said Harry.
Lastly, the IP capability of the ESInet has set the foundation for other NextGen 9-1-1 data services and applications, such as text-to-9-1-1 and media as they become more readily available. “We are confident that the benefits of this new technology will improve public safety in the near future,” acknowledged Harry.
For the past year or so we have been working to create a microwave network to provide a backbone for 9-1-1 IP based phone system and a Motorola 800 mhz trunked radio system. Its gone from a tiny project to a very large project. The parts have been ordered and here some of them are. I will include more details when I am allowed. Here are the back bone core routers we will use Cisco ASR 1002-X’s. We are using for MPLS-VRF. The WAN has 8 locations and will contain a variety of traffic from 9-1-1 VOIP traffic to 800 mhz trunked radio for Public Safety in Salt Lake County as well as video and data for the airport and other locations.
Update: September 21st 2013
We have installed 4 of the routers and have 1 microwave link up and operational. We have had several delays due to vendors but hope to have this project complete by mid October.
Routers Racked and Ready to Configure
Configuration Party with all the agencies networking people.
Installing our first router in Unified Police’s server room. We used this router to get their new Viper Phone system attached to VECC’s when their phone system came online.
Finally the vendor is installing the microwave dishes and radios.
Sometime in November it became evident to us that our network time clock was going the way of the Dodo. We were notified that the WWVB radios were going off line January 21, 2013.
Our old 8182 NetClock/2 system was based off old technology utilizing the WWVB signal (radio) out of Fort Collins, CO. The new current standard for a NetClock system is to utilize GPS as a timing reference. GPS requires a different antenna system. GPS is more accurate and reliable than the old WWVB signal. The newer GPS based NetClock’s offer legally traceable time; GPS provides a UTC world time reference.
I contacted Spectracom to find out what system would work for our complex time sync situation. We provide time to three different key components of 911 (phone, radio and computer network). The old NetClock required a main clock and secondary boxes to provide time to more than one network. We figured out quit quickly with the help of Matt Loomis which system we needed. We made the purchase and about two weeks later we had everything we needed to upgrade our time.
9483 NetClock is a new product release for Spectracom – (a direct replacement for the 8182 NetClock/2). This NetClock 9483 is a 5th generation GPS time server/master clock. It employs an all-new modular design, compared to our previous monolithic models, and offers several new features and capabilities we have been asked to offer in our timing platform. A key feature is the ability to add a 3-network port card module that would provide a total of 4 separate isolated network ports in a single 1U chassis to deliver our highly accurate NTP legally traceable timing.
Key Features and Benefits of the new NetClocks include:
Installing the new NetClock 9483 was fairly straight forward. First we ran the GPS antenna to the roof. Lucky for us our server room houses a lot of equipment that requires antennas on the roof. We ran the thick coaxial cable to the roof and mounted the GPS antenna. Next we mounted the NetClock to our rack and ran 2 CAT5 runs from our phone system network, and our computer network. We configured the IP addresses using the NetClock’s interface. We then used a laptop to configure all of the setup of the NetClock 9483. It was mostly straightforward but their web interface leaves a lot to be desired. It’s not very pretty and not user friendly enough. Not really an issue when this is a set it up once kind of thing. The final configuration required one phone call to tech support and was completely resolved in a matter of minutes.
We also installed a large clock on our operations floor. The clock is referred to as “God’s Clock” since every thing we do is based of its time.
Here we are more than 2 months later and the NetClock is functioning perfect and we didn’t have a single hiccup since the deactivation of the old NetClock system and radios.
The install took 2 of us about 4 hours. This including running cables, mounting an antenna on the roof and assorted other install tasks. The kit was complete with everything from cables to antenna, wiring, grounding and more. No complaints. Cost was approximately $10,000.