Chapter IV: The Pre-Site Visit Phase

Chapter IV: The Pre-Site Visit Phase

Chapter IV: The Pre-Site Visit Phase

A. Introduction

This chapter outlines the activities State agency staff need to complete in order to prepare for the on-site phase of the AAR. During this phase the State should identify those areas it wants to specifically address during the site visit. The State agency should view the AFCARS Assessment Review as an opportunity to examine its foster care and adoption data concerns completely and take advantage of the intensive technical assistance available from the Children's Bureau. The State should also review its CFSR data profile to identify any areas in which it may want to request assistance from the Federal AFCARS review team.

The pre-site visit phase involves the following activities:

documentation exchange;
test case scenario entry, extraction, and transmission;
case file review preparation; and
finalize the on-site agenda.
Back to Top

B. Overview of the Pre-Site Visit Activitites

The pre-site visit phase officially starts when a State is notified of its AAR. Ideally, a State will know by July that it will have its AAR the next Federal fiscal year. A State will have no less than three months notice of an AAR.

Once the dates are selected for the AAR, a confirmation letter will be sent to the State. This letter contains the following information:

The date of the on-site review.
The date the State's system documents are due to the Children's Bureau.
The date the case file sample and test cases will be sent to the State.
The date the test case extraction is due to the Children's Bureau.
The contact information for the Federal review coordinator and members of the Federal team.
Back to Top

C. Documentation Exchange

This section describes the documents that are exchanged between the State and the Children's Bureau. It is not the Children's Bureau's expectation that the State create new system documentation for the review team to analyze. The State should submit its technical documentation in its current format. Exhibit 4 contains examples of the types of documents to submit. The documentation should reflect the options available to the caseworkers for each AFCARS data element, and the State value for that option.

The State's system documentation may be submitted earlier than the due dates. States should begin consolidating the system documentation as soon as possible and maintain an up-to-date library of system documentation. This is helpful for those States that are developing a SACWIS and to those preparing for the SACWIS Assessment Review.

The Federal review team needs a fixed" set of documentation in order to prepare its analysis and provide the State with useful feedback. The State should submit its most current version of the extraction program code and reference tables. If the State identifies areas that need to be corrected, or is in the process of making overall system modifications after the documentation is submitted to the Federal team, then the State should provide the Federal team (either onsite or during the pre-site phase) a list of all changes and planned changes. When modifications are subsequently made to the screens and/or the program code as it relates to AFCARS those modifications will be included in the post-site visit review.

Procedure

By the due dates, submit copies of the documentation to the Children's Bureau. The documentation is to be submitted electronically.
E-mail all documents to the Federal team members.
If electronic copies of the documents are sent on a compact disk, this must be sent via overnight mail to the Children's Bureau4. Do not send zip disks.
If the State technical staff has questions regarding what technical documentation is to be submitted, contact the Federal Technical Coordinator.
Screen prints must include a copy of the screen as it first appears on the system, and copies of AFCARS related drop-down lists. This may take more than one screen print. The Federal review team must have the actual view that appears to the user.
A data dictionary should include all of the State's values/codes used in the system. While many of these are not related to AFCARS, we have found it is easier for the State to provide a comprehensive list. This provides the Federal team an opportunity to review the values and ensure that the State is including all appropriate values in its AFCARS extract.
If the extraction code is sent to the Federal team in multiple files that are sections of the complete code, include a master list that identifies the order the files are executed.
States that use forms as a means to collect the AFCARS data are to include these in their documentation submission.
Exhibit 4: State Documentation List

Programming Logic (Source Code): The programming logic that derives and extracts each AFCARS element from the State's System.
Data Element Dictionary: A data dictionary or similar documentation will provide the Federal team with definitions and representations of the State's values for AFCARS -related fields. This document should be complete and reflective of options available to the caseworker to select. This documentation may be supplemented by code/lookup tables that will enable the team to understand elements referenced in the source code or mapping forms. For example, placement code 257 = relative foster care, licensed.
Relevant Reference Tables and Fields: Specific fields and tables from which the data are pulled.
Database Structures: Tables referenced in mapping documentation.
Extraction Routine Notes: A logical description, or the steps in pseudo code, of how records are selected for inclusion in the foster care and adoption submission.
Data Entry Screens: Printouts of screens relevant to AFCARS data elements.
Change Request Orders: Changes that the State has planned that impact the AFCARS data elements that are not included in the current version of the extraction program code.
Organizational Charts: Charts of the program and information technology divisions

Note: It is crucial that the documentation be submitted to the Children's Bureau on time. Delays in the submission of the documents affect the Federal team's preparedness for the on-site review and may result in an incomplete on-site evaluation.

Back to Top

D. Test Case Scenarios

The use of test cases is a way to assess the State's collection and extraction of the AFCARS data. This process assists the Federal review team to gain a better understanding of the State's policies, practices, training, and extraction routine prior to the onsite review. The records that are created by the test cases are extracted from the State's data system and compared with an established answer key. This information in conjunction with the analysis of the extraction program code and the other components of the AAR, provides the Federal team with a comprehensive view of the State's system.

The intent of the test cases is to validate the State's information system for accuracy. As such, the Federal review team coordinator may answer some questions the State has on the cases, but most will not be answered in the interest of determining objectively the State's AFCARS capability.

Procedure

The Federal review coordinator e-mails the test cases to the State review coordinator by the established due date.
The State review coordinator will identify a staff member who is familiar with the system and/or case practice to enter the cases. Approximately 40 working hours should be allotted for this task.
A training or test environment may be used to enter the test cases. This should mirror the production system. The State needs to ensure that the method used to enter the data will also allow for the entry/extraction of the financial data elements that are obtained through an interface.
Once the test cases are entered into the system, the State review team coordinator will contact the Federal Technical Coordinator for instructions on electronically transmitting the test file(s). The file must be submitted by the established due date.
The electronic copy of the scenarios are to be transmitted to the Federal review coordinator (in Word) with the encrypted case number provided for the respective test case. Additionally, the State team can include any relevant notes pertaining to the test cases as necessary.
Back to Top

E. Case File Review Preparation

Another tool used in the AFCARS Assessment Review process is the case file review. A sample of 80 foster care records and 30 adoption records is pulled for the AFCARS reporting period under review. The AFCARS data submitted to the Children's Bureau on each record is then compared to information found in the paper case file during the onsite review. The process involves all members of the State and Federal teams, technical and program. The State is encouraged to incorporate field staff, including supervisors and staff from training units, etc., as part of the State team for the purpose of reviewing cases. See Chapter III for additional information on who the State may want to include on the case file review team. A preliminary analysis of the findings is conducted during the onsite review and shared with the State. A further discussion of the onsite process can be found in Chapter V.

The Children's Bureau has found that while there may be challenges to identifying the information in the paper file, the process provides very valuable information to the review teams. The case file review is also the only means for the Federal team to assess the accuracy and the level of completeness of the State's conversion process from a paper or legacy system to its new information system. The findings often provide additional information that increases the Federal team's understanding of the data reported to AFCARS. Also, this process allows the review team to assess how well records are being kept up-to-date, the accuracy of the AFCARS data, and usage of the State's information system. Typically, this process does not identify new problems, but confirms findings from the other components of the AAR.

While the Children's Bureau recognizes that current data may no longer be in the paper files but will, instead, be in the automated information system, there are some documents that may not be part of the State's information system, such as medical reports, court reports, home studies, etc. These documents usually provide a significant amount of the information for the case file reviewers. It is important that the State have available the complete case file; frequently relevant information is found in a sibling's folder instead of the folder on the child under review which can be challenging.

Since the case file review is the only means to assess conversion, the cases selected for the review will primarily be those in which the most recent removal date, or the first removal date, precedes the date the State's system went operational. If the State phased in its operational status, then the sample may reflect these dates.

Depending on the length of time since the State converted to a new information system, this portion of the AAR may be modified to accommodate the State's situation. During the initial conference call, the Children's Bureau and the State will discuss and plan for the case file review and make adjustments, as necessary.

Procedure

Once the State review team coordinator receives the list of encrypted or sequential case record numbers from the Federal review team coordinator, he/she should:

Create a list to cross-reference the encrypted/sequential record number with the actual case record number.
Arrange for the case files to be delivered to the State office by the first day of the on-site review.
Ensure that case records are as complete and up-to-date as possible, and include any files maintained separately, e.g., child protective services files, provider files, and eligibility files. All case records should include:
court orders;
placement history;
caretaker (person from whom child was removed) information;
medical history;
biological parent file for the adoption records; and
foster care records on children adopted (if the relevant AFCARS information cannot be found in the child's adoptive record).
Note Regarding Sealed Adoption Records: If a sufficient number of cases cannot be identified for review due to their having been sealed, then the State will have to petition the court to open the records prior to the on-site review.

Identify and schedule staff to participate in the case file review. It is recommended that the State have field workers, supervisors, and/or regional administrators participate in this part of the review.
Back to Top

F. The On-Site Agenda

Appendix E contains a sample agenda for the on-site review. Any changes to the overall agenda must be discussed with the Federal review team coordinator prior to any changes being made to the agenda. The State review team coordinator should provide the Federal review team coordinator with a list of the names and titles of the State staff participating in each segment of the AAR.

Back to Top

G. Site Logistics

The on-site review activities are conducted in one location. The State capitol is generally the designated location since it is the location of the State agency central office/administrative staff, but another site may be mutually agreed upon. It is preferable to conduct the meetings in one large room to allow enough space for the size of the team and materials needed for the AAR. A separate room can be added on the day of the case file review, as this activity will include additional staff and the case files. The room used for the case file review should be secure to protect the confidentiality of the case records.