Viewpoint_DataLab_1600_1610-Update (1)

Prévia do material em texto

Teradata Viewpoint and Data Lab 
Introduction and Update (16.00 & 16.10)
Gary Ryback – Product Management
June 2017
‹#›
1
‹#›
Teradata Viewpoint
Overview
16.00 New Content
16.10 New Content
Teradata Data Lab
Overview
16.00 New Content
16.10 New Content
Agenda
2
‹#›
Teradata Viewpoint
Overview
‹#›
‹#›
Teradata Viewpoint 
INTEGRATED DATA WAREHOUSE
 TERADATA UNIFIED DATA ARCHITECTURE
DATA
 PLATFORM 
HADOOP
TD Appliance
Or
*Commodity
Teradata Database
Integrated Discovery platform
Teradata Aster Database 
Viewpoint
TVI
MDM
Teradata Viewpoint
w/Viewpoint 16.10
UDA Monitoring
Multiple Platforms
Varying release versions
Browser & Mobile access
Snap-shot architecture
Cache DB data repository
Self-Service
For Everyone!
LDAP based authentication
Auto-provisioning +
Role based security
‹#›
Auto-provisioning for not only automated user addition but also role mapping and default dashboards
Hadoop commodity requires additional licensing; Teradata Hadoop Appliance can be monitored for no additional cost
© 2014 Teradata
‹#›
Multi-Temp
Data Lab
DSA / BAR
Workload Management Strategies
DATA MOVER
Unity
ECOSYSTEM 
MANAGER
Teradata Database
Teradata ASTER 
The Viewpoint Portlets Universe
Teradata QueryGrid™ 
Presto
Server Management 
‹#›
This slide just shows the various products and platforms that are supported with Viewpoint portlets. Some are part of the core portlets (i.e. QueryGrid or Hadoop), others are added when the associated solution is implemented (i.e. Data Mover or Unity).
© 2014 Teradata
‹#›
Viewpoint Single-System & Multi-System Platforms
Teradata Viewpoint TMS
DM
KVM
VP
KVM
EM
KVM
Teradata “Consolidated” VP, EM, DM TMS
Teradata VMS Viewpoint
Note: Viewpoint 16.00 & 16.10 instances only run on SLES11 SP3 OS
‹#›
Supported Systems and Products Summary
Monitored Systems:
Aster Database
Hadoop
Cloudera
Hortonworks
Managed Systems
Presto	(VP16.00)
QueryGrid 	(VP16.00)
Teradata Database
Add-on Product Portlets:
BAR / DSA
*Data Lab
Data Mover
Ecosystem Manager
Server Management
Unity
*Note: Teradata Data Lab is a separately sold product
‹#›
Teradata Viewpoint
16.00 and 16.10 New Content
‹#›
‹#›
GA November 2016
Key New Features: 
Support for Teradata Database 16.00 release
Workload Management and Stats Manager
QueryGrid and Presto as new “Monitored Systems”
QueryGrid 2.0 Configuration and Monitoring
Presto Monitoring
Lock Viewer Portlet Improvements
System Support: 
Teradata Database 14.00, 14.10, 15.00, 15.10, 16.00 releases
Aster Database 6.00, 6.10, 6.20
Hortonworks Hadoop 2.3, 2.4, 2.5
Cloudera Hadoop 5.5, 5.7, 5.8 
QueryGrid 2.0
Viewpoint 16.00 Release
‹#›
Workload Management – Flex Throttles
Prevents system under-utilization by allowing certain workloads to exceed the throttle limits at certain times.
Added two flex events :
Available ATWs
CPU Utilization
Support for Teradata Database 16.00 Release
‹#›
Flex throttles allow you to use available system resources for high priority workloads. It comes with an evaluation flag, so you can “preview” the changes before they are actually applied.
You control what it means to have system resources available; AWTs, CPU, or both
You control how much to increase concurrency
You control which workloads, and which state, to participate in Flex throttles
Alerting is available. 
AWT availability is a key concept in TASM. Use to trigger events, State, Resource Throttles, and now Flex Throttles. This flag lets you control how you define AWT availability.
© 2014 Teradata
‹#›
Stats Manager
New ability to monitor progress of an analyze job
Set time limit 
All long running Analyze jobs 
For a specific Analyze job
Support for Teradata Database 16.00 Release
‹#›
Stats manager is an application that lets you manage the analysis and collection of jobs in your Teradata Database. 
These jobs can run for a long time, as they are analyzing DBQL data. New in 16.00 (also backported to 15.10), we have job level monitoring for Analyze jobs. Collect jobs had monitoring earlier, now made consistent with Analyze jobs.
© 2014 Teradata
‹#›
QueryGrid 2.0 Configuration
New QueryGrid Portlet
Administer QueryGrid components like Fabrics, Connectors, Links, Networks, Communication Policies
Test configuration changes before applying them in production
Perform software updates and diagnostic checks
Define and view alerts on QueryGrid systems, QueryGrid manager or collectors
‹#›
QueryGrid 2.0 Monitoring
Updates to Query Monitor Portlet
Enhanced query monitoring capabilities:
Before: On individual source or target systems
Now: Follow a query across multiple systems
See details for QueryGrid operations in addition to query execution details on each system
Monitor QueryGrid operation for systems that are not currently monitored in Viewpoint
‹#›
Viewpoint Query Monitoring capabilities enhanced to support QueryGrid query monitoring
Currently, Viewpoint provides query monitoring capabilities on individual source or target systems
With Viewpoint 16.00, you can now follow a query across multiple systems and you can see details for QueryGrid operations in addition to query execution details on each system
QueryGrid operation monitoring is also available for systems that are not monitored in Viewpoint (pre 16.00 version).
© 2014 Teradata
‹#›
Viewpoint 16.00 - Presto Monitoring 
‹#›
Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. Presto is the target technology for querying Hadoop for Teradata QueryGrid.
Integrate monitoring of Presto into Viewpoint
Presto is the target technology for querying Hadoop for Teradata QueryGrid™ 
Monitoring Presto query execution through Viewpoint provides a complete view of executing queries through Teradata QueryGrid™.
© 2014 Teradata
‹#›
Lock Viewer
Now supports filtering and sorting on all columns
New blocked session details view:
Viewpoint 16.00 – Lock Viewer Portlet Improvements
‹#›
GA July 2017
Key New Features: 
Support for Teradata Database 16.10 release
MAPS Manager portlet – Teradata MAPS (Multiple Hash Maps)
Workload Management - New TASM IO event 
Viewpoint Improvements
QueryGrid enhancements
Space Usage now displays more object types
Other Enhancements
Monitored Systems: Offers a more descriptive system name (up to 30 chars) 
Alert Notification Service supported with Windows Server 2012 R2 & 2016 
Support for Microsoft Edge Browser
Viewpoint 16.10 Release
‹#›
© 2014 Teradata
‹#›
GA July 2017
System Support: 
Teradata Database 14.00, 14.10, 15.00, 15.10, 16.00, 16.10 releases
Aster Database 6.10, 6.20
Hortonworks Hadoop 2.4, 2.5	(at GA)
HDP 2.6 post GA
Cloudera Hadoop 5.8, 5.9		(at GA)
CDH 5.11 post GA
QueryGrid 2.0
Notes:
Hadoop compatibility will be updated as new versions release
Aster AX monitoring planned with Viewpoint 16.20
Viewpoint 16.10 Release
‹#›
New MAPS Manager Portlet
Requires Teradata Database 16.10 or newer 
Listed in the Viewpoint “Applications” grouping
Viewpoint 16.10 – MAPS Manager
The MAPS Manager portlet provides two methods to move objects to a new map enabling improved system performance without extended downtime.
System Monitoring - Provides ongoing recommendations to optimize the location of objects. One can accept and schedule object moves when convenient.
Object Workflows - Object workflows lets one create ad-hoc workflows where one can select objects and designate the order and time for movement.
‹#›
© 2014 Teradata
‹#›
Viewpoint 16.10 – MAPS Manager: System Monitoring
Periodic, ongoing monitoring providing recommendations at system level
Bucketed view of recommendations
Status updates and scheduling provided for accepted recommendations
‹#›
© 2014 Teradata
‹#›
Viewpoint 16.10 – MAPS Manager:Object Workflows
‹#›
© 2014 Teradata
‹#›
Object 
Selection
Workflow Wizard is used to: select objects to analyze, schedule analysis, and setup alerts
Analyze
Objects
From Workflow Details Analyze SP is called. The MAPS feature generates recommended actions, and reports errors
Evaluate Recommendations
In Workflow Details you evaluate recommendations… Include/exclude objects to move, change target map, order and grouping
Move Objects
From Workflow Details Move SPs are called. MAPS Manager reports progress and errors. Also supports aborting gracefully or immediately
Teradata Database 16.10 – New TASM I/O event 
Implementation of a new TASM event for I/O consideration. 
As part of the State Matrix:
‹#›
TASM has a new event based on I/O utilization levels. For example, a user might want to take an action if the I/O bandwidth was 90% saturated for some period of time
© 2014 Teradata
‹#›
Viewpoint 16.10 – QueryGrid Enhancements
With Viewpoint 16.00, a new QueryGrid portlet was added for configuration and integrated monitoring within the Query Monitor portlet. With Viewpoint 16.10, there are new enhancements to both configuration and monitoring.
Support for Bridges - Bridge systems are sets of nodes that act as routers for data movement. They can be added to a QueryGrid fabric to offload computationally intensive operations, such as data compression or encryption. To use a bridge system, it must first be defined within the QueryGrid portlet Systems link. 
New Fabric Overview – A new more intuitive visualization of connectivity within the QueryGrid portlet that allows hovering for component status and an easy click to the detailed view. Further for very large fabric, there is a mini-map navigator.
‹#›
© 2014 Teradata
‹#›
Viewpoint 16.10 – QueryGrid – Support for Bridges 
Configuration of the System and Bridge.
‹#›
© 2014 Teradata
‹#›
Viewpoint 16.10 – QueryGrid – Support for Bridges 
Query Monitor portlet monitoring for Bridges
‹#›
© 2014 Teradata
‹#›
Viewpoint 16.10 – QueryGrid – New Fabric Overview 
A new more intuitive visualization within the QueryGrid portlet.
‹#›
© 2014 Teradata
‹#›
Viewpoint 16.10 – QueryGrid – New Fabric Overview 
View for large QueryGrid fabrics
‹#›
© 2014 Teradata
‹#›
Space Usage: 
Now displays other object types (View, Macro, Authorization, Trigger, Instance Or Constructor Method, and User Defined Data Type.
Viewpoint 16.10 – Space Usage Portlet Improvement
‹#›
© 2014 Teradata
‹#›
Viewpoint 16.10 – Other Enhancements
Monitored Systems: Offers a more descriptive system name (>8 chars)
Increased from 8 to 30 characters
Alert Notification Service now supports Windows Server 2012 R2 & 2016
Maintained Windows Server 2008 R2
Microsoft Edge Browser
Default w/Windows 10 OS replacing Internet Explorer as the default web browser
‹#›
© 2016 Teradata
‹#›
Teradata Data Lab
Overview
‹#›
‹#›
Analytical Community
Database Administrators
Intelligent Discovery
‹#›
Self Service 
	Governance
Self Management
Workload Management
Production Sandboxes
Process
Discovery
‹#›
The Common Result
3_89
ODS
IDW
mart
Excel
 files
SAS
 data
‹#›
Self Service - Community
3_89
Self Service
‹#›
I need to be able to quickly get a production infrastructure bypassing the normal implementation process
I want to easily share my lab with others I collaborate with
I need flexibility for data lab growth and longer discovery duration
I need to be able to create and load tables with my discovery data
I want email notifications for important actions
© 2014 Teradata
‹#›
Speed
Agility
Flexibility
Ease of Use
3_89
Self Management - Community
Teradata Database
Marketing 
Lab Group
Campaign
Lab
Data Scientist
Lab Group
Sales 
Lab Group
Promotion
Lab
Risk Analytics
Lab
EMEA
Lab
APCJ
Lab
Americas
Lab
Customer 
Segmentation Lab
Jills Personal
Lab
‹#›
Self Management
3_89
Data Lab Owner
Owns and manages the Data Lab
Receives Data Lab email notifications
Lab Group Owner
Approval authority for Lab Group requests
Receives Lab Group email notifications
Member
Performs data analysis
Examples:
Business Analyst
Statistician
Data Scientist
IT Application Developer
DBA
Architects the Lab Group
Configures workload management
Dan 
the DBA
Sam the Power User
Ann the Business Analyst
‹#›
3_89
Self Management
‹#›
Analytical Community
Database Administrators
Intelligent Discovery
‹#›
Self Service 
	Governance
Self Management
Workload Management
Governance - DBAs
‹#›
I need controls for the amount of space that can be consumed
I need to manage how long a discovery environment can exist
I don’t want the headache; clean-up must be automated
I need an easy way to view who has access to what
© 2014 Teradata
‹#›
Concerns
Space
Clean up
Access
Duration
Governance
3_89
@Lab Group level:
Fixed space
Lab Group owners
Object permissions 
Access reports 
@Data Lab level:
Space management 
Expiration
Deletion
Request limit of maximum size
with auto reject option
Request limit of maximum cumulative life
with auto reject option
‹#›
Workload Management
3_89
Execution Phase
Prioritization
Exceptions
Classification, State Matrix, 
and VP’s
Partitions
Workload
State Matrix
Pre-execution Controls
Filters
Throttles
(“Lab Prefix” + User’s Data Lab Name)
i.e. “dlab_lost-cart”
‹#›
What tools do I use with Data Lab?
‹#›
Teradata Data Lab
16.00 and 16.10 New Content
‹#›
‹#›
GA November 2016
Key New Features: 
New Permissions Management
Mobile Approvals
Expiration Request Options Management
Always “reject” option for request limits
Requires Viewpoint 16.00
Data Lab 16.00 Release
‹#›
New step in the Lab Group creation process:
Modifiable checklist for Teradata Database permissions provides flexibility into what additional permissions can be granted
 
Data Lab 16.00 - Teradata Database Permissions 
‹#›
Prior to Data Lab 16.00, the Teradata Database access permissions were set by Data Lab and did not allow flexibility into what additional permissions could be added. Data Lab 16.00 adds a new step in the Lab Group creation process (as well as in editing) that provides the ability to toggle many more Teradata permissions. The additional full access lab permissions for default users and lab users are:
Create table
Statistics
Show
Alter procedure
Execute procedure
Alter function
Execute function
Create external procedure
Alter external procedure
© 2016 Teradata
‹#›
If your Viewpoint user has permission to view the Approvals tab in the Data Labs portlet, you will now be able to approve/deny pending requests from within the mobile version of Viewpoint
Data Lab 16.00 – Mobile Request Approvals
‹#›
Data Lab 16.00 – Expiration Request Options Management
Enhancement to lab expiration request options management:
New feature sets maximum extension period for any particular extension requests
Expiration pick list limits the extension period to less than the pre-set maximum
‹#›
Prior to Data Lab 16.00, an extension request for a lab utilized a pick list of options between 30 and 365 days as well as the option to customize the number of days to whatever was desired (an “x” number of days). 
Data Lab 16.00 offers a maximum extension setting for any particular extension requests. This new feature limits the options presented during a extend lab request to a certain maximum. When this new setting is set, the provided pick list would only offer those options less than the set maximum and would not allow any custom settings greater than that as well.
© 2016 Teradata
‹#›
Data Lab 16.00 – Request Limits “AlwaysReject” option
Existing feature controls maximum setting controls for duration (how long) and size (how big) for data labs within a lab group
As part of the Lab Group setup, a new “Always Reject” option allows bypassing the manual approval/ rejection actions 
‹#›
Data Lab 16.00 enhances the “Request Limits” feature (introduced in 15.10) where one can implement maximum setting controls for both the duration (how long) and size (how big) for data labs within a lab group.
The behavior with Data Lab 15.10 was, regardless of automated approval settings, the requests exceeding one of these limits would be automatically sent for approval. With 16.0, an “Always Reject” option allows to bypass this behavior if desired. 
© 2016 Teradata
‹#›
GA July 2017
Tables view changed to Objects view
As it now displays more object types
More information for requesters and approvers
Add the Request ID and Location to the Requests tab 
Add the Requester Email to the Approvers tab 
Include Requester and Requester Email as part of the request email 
Ensure proper permissions for a Lab Group deletion
Requires Viewpoint 16.10
Data Lab 16.10 Release
‹#›
© 2014 Teradata
‹#›
Tables view is now Objects view: 
Now displays other object types (View, Macro, Authorization, Trigger, Instance Or Constructor Method, and User Defined Data Type.
Data Lab 16.10 – Objects View
‹#›
© 2014 Teradata
‹#›
Data Lab 16.10 – More info for requesters and approvers
Allowing for more informed decisions: 
Request ID and Location (shows lab-prefix) to the Requests tab 
Requester Email (more descriptive than just login)to the Approvers tab 
Requester and Requester Email as part of the request email 
‹#›
© 2014 Teradata
‹#›
Data Lab now verifies necessary permissions for a Lab Group deletion 
No more persisting strike-through Lab Groups
Data Lab 16.10 – Permissions for a Lab Group deletion
‹#›
© 2014 Teradata
‹#›
‹#›
‹#›
‹#›